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EXECUTIVE SUMMARY 


Animal-vehicle collisions affect human safety, property, and wildlife, and the number of 
animal—vehicle collisions has been increasing in many regions across North America. For this 
project The Western Transportation Institute at Montana State University (WTI/MSU) evaluated 
a relatively new mitigation measure aimed at reducing animal—vehicle collisions while allowing 
animals to continue to move across the landscape. WTI/MSU evaluated different types of animal 
detection systems from different manufacturers with regard to system reliability and operation 
and maintenance aspects. Animal detection systems detect large animals (e.g., deer, elk, moose, 
or pronghorn) as they approach the road. When an animal is detected, signs are activated 
warning drivers that large animals may be on or near the road at that time. Previous research has 
shown that, depending on road and weather conditions, the warning signs can cause drivers to 
reduce their speed. Warning signs may also result in more alert drivers, which can lead to a 
substantial reduction in stopping distance: 20.7 m (68 ft) at 88 km/h (55 mi/h). Finally, research 
from Switzerland has shown that animal detection systems can reduce ungulate—vehicle 
collisions by as much as 82 percent. 


The main objective of this project was to evaluate the reliability of different animal detection 
systems from different manufacturers at the same site under similar circumstances and to 
recommend minimum standards for system reliability. A test facility (Roadside Animal 
Detection System (RADS) test-bed) was constructed near Lewistown, Montana. Nine different 
animal detection systems from five different manufacturers were installed to detect horses and 
llamas that roamed in an enclosure. Data loggers recorded the date and time of each detection for 
each system. The animal movements were also recorded by six infrared cameras with a date and 
time stamp. By analyzing the images and the detection data, researchers were able to evaluate the 
system for a variety of reliability parameters. 


The results of the reliability tests showed that different detection technologies detect large 
animals more or less frequently as an animal passes through the detection area or line of 
detection. The percentage of false positives (i.e., a detection is reported by a system but there is 
no large animal present in the detection zone) and the average number of false positives per hour 
was relatively low for all systems (<1%; <0.10/hr). The percentage of false negatives (1.e., an 
animal is present in the detection zone but a system failed to detect it) and the average number of 
false negatives per hour was highly variable (0—31%; 0-1.61/h) (all types of false negatives 
combined). The percentage of intrusions (1.e., animal movements across the detection line) that 
were detected varied between 73 and 100 percent. The results suggest that some animal detection 
systems are quite reliable in detecting large mammals with few false positives and false 
negatives, whereas other systems have relatively many false negatives. 


The reliability of animal detection systems is influenced by a range of environmental conditions. 
High winds were associated with an increase in different types of false negatives for most 
passive infrared area-cover systems (i.e. systems that detect an animal within a certain range of a 
sensor, mostly through passive infrared or radar technology). High winds were associated with 
both an increase in false positives and a decrease in false positives for different types of systems, 
suggesting that passive infrared area-cover systems become less sensitive with high winds 
whereas break-the-beam systems (i.e. systems that detect an animal when the animal blocks or 
reduces a signal (active infrared, laser or radar) transmitted by a sensor and received by another 
sensor) that rely on a very narrow beam may start generating false positives, presumably because 
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the sensors sway slightly in and out of alignment. The latter suggests the importance of a stable 
foundation and pole for break-the-beam systems. Stable foundations and poles may also be 
beneficial to passive infrared area-cover systems, but it is unclear if the increase in false 
negatives for such systems is caused by movement of the sensors that tend to be higher up on a 
pole than sensors for break-the-beam systems, or by vegetation or pockets of hot and cold air that 
move in the wind across the detection zone. The effects of wind direction are hard to interpret, 
but it may be that winds oriented perpendicular to the systems caused vegetation or pockets of 
hot and cold air to trigger systems more often than winds oriented more parallel to the systems. 
Higher temperatures are generally associated with higher error rates. This could be due to 
temperature causing reduced performance of the equipment. In addition, passive infrared systems 
may not be able to distinguish clearly between pockets of hot air and moving animals. However, 
higher temperatures are concentrated in time (summer) and it is possible that factors other than 
temperature caused more errors in summer. Animal behavior and possible effects on the 
likelihood of correct detections and errors may have also been influenced by temperature. Three 
systems had fewer false negatives during the night compared to during the day. This may be 
related to lower temperatures or higher contrasts in temperatures of the animals and their 
surroundings during the night. However one system had more false negatives during the night 
compared to during the day. Excellent visibility was associated with fewer false positives for a 
break-the-beam system, which suggests that relatively low visibility may block or reduce the 
narrow signal path of optical break-the-beam systems. It is unclear why excellent visibility was 
associated with an increase in false negatives for one of the area-cover systems. Precipitation 
was rarely observed during the test periods and its effect on system reliability is unclear. 
However, higher relative humidity was mostly associated with an increase in errors, and to a 
lesser extent with a decrease in errors. Finally, llamas were substantially harder to detect for 
most systems, especially passive infrared area-cover systems, than horses, probably because of 
their smaller body size. 


Three stakeholder groups—employees of transportation agencies, employees of natural resource 
management agencies, and the traveling public—were surveyed with regard to their expectations 
on the reliability and effectiveness of animal detection systems. There was considerable 
agreement in the responses of the three groups. Based on the results from the survey, the 
researchers recommend the following performance requirements for the reliability and 
effectiveness of animal detection systems: 


e Animal detection systems should detect at least 91 percent of all large animals that 
approach the road. 


e Animal detection systems should have fewer than 10 percent of all detections be false. 


e Animal detection systems should result in at least 71 percent reduction of wildlife— 
vehicle collisions. 


The recommended performance requirements for the reliability of animal detection systems were 
compared to the results of the reliability tests. Five of the nine systems tested met the 
recommended performance requirements for reliability. However, experiences with installation, 
operation and maintenance showed that the robustness of animal detection systems may have to 
be improved before the systems can be deployed on a large scale. 


This report also presented a concept of operation and a review of Intelligent Transportation 
System (ITS) architecture and infrastructure for animal detection systems. Currently, roadside 
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animal detection systems present drivers with warnings displayed on road signs. In the future, 
roadside animal detection systems may also transmit warning signals to traffic approaching a 
location where a large animal has been detected on or near the road. This procedure would 
require a two-way GPS-based communication system. With animal detection system 
deployments becoming more numerous, standards for communication and ITS integration will 
have to be further developed and accepted. 


Finally, the researchers reviewed seven sites in Montana for the potential installation of an 
animal detection system. 


Based on the results of the study, the researchers concluded: 


e Different detection technologies detect large animals more or less frequently as an animal 
passes through the detection area or line of detection. This implies that care must be taken 
in evaluating the reliability of different technologies, and in comparing them to other 
systems or minimum performance requirements. 


e The percentage of false positives and the average number of false positives per hour was 
relatively low for all systems (<1%; <0.10/hr). False positives do not appear to be a major 
concern with regard to the reliability of animal detection systems. 


e The percentage of false negatives (all types of false negatives combined) and the average 
number of false negatives per hour under the test circumstances was highly variable (0— 
31%; 0-1.61/hr). The percentage of intrusions (i.e., situations where at least one animal 
was present in the detection area) that were detected varied between 73 and 100 percent. 
The results suggest that false negatives are a major concern for some animal detection 
systems, but not for others. 


e Environmental conditions influence the reliability of animal detection systems. Therefore 
the environmental conditions at a site should be carefully evaluated before selecting a 
suitable system. In addition, since the size of the species affects the reliability of some of 
the systems, it is also important to consider the size target species and how that may 
affect the reliability of a particular system. Besides system reliability, system robustness 
(i.e. consistent performance over time, low monitoring and maintenance effort), size of 
the equipment (landscape aesthetics), and the road length that the sensors are able to 
cover needs to be considered. 


e The recommended performance requirements for the reliability of animal detection 
systems were compared to the results of the reliability tests. Five of the nine systems 
tested met the recommended performance requirements for reliability. However, 
experiences with installation, operation, and maintenance show that the robustness of 
animal detection systems may have to be improved before the systems can be deployed 
on a large scale. 


e Currently, roadside animal detection systems present drivers with warnings displayed on 
road signs. In the future, roadside animal detection systems may also transmit warning 
signals to traffic approaching a location where a large animal has been detected on or 
near the road. With animal detection system deployments becoming more numerous, 
standards for communication and ITS integration will have to be further developed and 
accepted. 
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1. INTRODUCTION 


Author: Marcel P. Huijser, Western Transportation Institute, College of Engineering, Montana 
State University 


1.1. Background 


Animal—vehicle collisions affect human safety, property, and wildlife. In the United States, more 
than 90 percent of animal—vehicle collisions involve deer (Hughes et al., 1996), with the total 
number of deer—vehicle collisions estimated at more than one million per year (Conover et al., 
1995). These collisions were estimated to cause 211 human fatalities, 29,000 human injuries, and 
over $1 billion in property damage a year (Conover et al., 1995). These numbers are likely to 
have increased even further over the last decade (Hughes et al., 1996; Romin & Bissonette, 1996; 
Anonymous, 2003). In most cases, the animals die immediately or shortly after the collision 
(Allen & McGullough, 1976). In some cases, it is not just the individual animals that suffer; 
some species are also affected on the population level and may even be faced with a serious 
reduction in population survival probability (e.g., van der Zee et al., 1992; Huijser & Bergers, 
2000; Proctor, 2003). In addition, for some species a monetary value (e.g., hunting, recreation) is 
lost to society once an individual animal dies (Romin & Bissonette, 1996; Conover, 1997). 


Historically, animal—vehicle collisions have been addressed through signs warning drivers of 
potential animal crossings. In other cases, wildlife warning reflectors, mirrors or wildlife fences 
have been installed to keep animals away from the road (e.g., de Molenaar & Henkens, 1998; 
Clevenger et al., 2001). However, conventional warning signs appear to have only a limited 
effect because drivers are likely to habituate to them (Pojar et al., 1975). Also, wildlife warning 
mirrors or reflectors may simply not be effective (Reeve & Anderson, 1993; Ujvari et al., 1998). 
Furthermore, wildlife fences can isolate populations. Wildlife fencing has been combined with 
wildlife crossing structures to address these limitations (e.g., Foster & Humphrey, 1995; 
Clevenger et al., 2002) but, primarily due to their high cost, such crossing structures are limited 
in number and size. 


For this project, the Western Transportation Institute at Montana State University (WTI/MSU) 
evaluated a relatively new mitigation measure aimed at reducing animal—vehicle collisions while 
allowing animals to continue to move across the landscape. WTI/MSU evaluated different types 
of animal detection systems from different manufacturers with regard to system reliability and 
operation and maintenance aspects. Animal detection systems detect large animals (e.g., deer, 
elk, moose, or pronghorn) as they approach the road. When an animal is detected, signs are 
activated warning drivers that large animals may be on or near the road at that time. Previous 
research has shown that, depending on road and weather conditions, the warning signs can cause 
drivers to reduce their speed (see review in Huijser & McGowen, 2003; Kinley et al., 2003; 
Dodd & Gagnon, 2008). Warning signs may result in more alert drivers (Green, 2000), which 
can lead to a substantial reduction in stopping distance: 20.7 m (68 ft) at 88 km/h (55 mi/h) 
(Huijser et al., 2006a). Finally, research from Switzerland has shown that animal detection 
systems can reduce ungulate—vehicle collisions by as much as 82 percent (Kistler, 1998) or 81 
percent (Romer et al., 2003). Similar results come from Arizona (91 percent; Dodd & Gagnon, 
2008) and Montana (58-67 percent; Huijser et al., 2009). 
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While projects that evaluate individual animal detection systems remain valuable and continue to 
contribute to the existing knowledge, the ongoing development and implementation of animal 
detection system technologies can benefit by expanding beyond this limited scope. Huijser et al. 
(2006a) identified the remaining research questions for the emerging field of animal detection 
systems. Two of the most important questions that remain are how reliable the different animal 
detection systems really are and what the minimum standards for system reliability should be. In 
addition, the efforts and costs related to installation, operation, and maintenance are generally not 
available, and it is currently impossible to compare different systems from different vendors with 
regard to this important parameter. 


1.1.1. Related Studies 


Huijser et al. (2006a) listed all known animal detection system sites throughout Europe and 
North America. They also summarized experiences with the operation and maintenance of these 
systems. In addition, WTI/MSU has ongoing projects that evaluate the reliability and 
effectiveness of animal detection systems in roadside environments. WTI/MSU also documents 
driver opinions and experiences with operation and maintenance. One system, installed in the fall 
of 2002, is located along U.S. Highway 191 in Yellowstone National Park in Montana. This 
WTI/MSU Pooled Fund Study is funded by the Federal Highway Administration (FHWA) and 
15 departments of transportation: the Alaska Department of Transportation and Public Facilities, 
and the California, Indiana, Iowa, Kansas, Maryland, Montana, Nevada, New Hampshire, New 
York, North Dakota, Oregon, Pennsylvania, Wisconsin, and Wyoming Departments of 
Transportation. 


As of August 2006, only some of the vendors of animal detection systems have actually installed 
their animal detection systems at one or more sites (Huijser et al., 2006a). In addition, some 
systems that have been installed are not yet operational and others have been abandoned for 
various reasons (Huijser et al., 2006a). Few animal detection systems have been studied with 
regard to system reliability and system effectiveness. Examples include the area-cover systems in 
Switzerland (Kistler, 1998; Romer et al., 2003) and Finland (Muurinen & Ristola, 1999; Taskula, 
1999), the systems in Wyoming (Gordon et al., 2001; Gordon & Anderson, 2002) and the area- 
cover system in Kootenay National Park, Canada (Kinley et al., 2003). Some studies have not yet 
been completed (e.g., Huijser et al., 2006a), but most systems have never been evaluated 
properly, and the information with regard to those systems remains anecdotal at best. 


1.2. Project Outline 


In order to select the most reliable animal detection system and to gain insight in the cost-benefit 
ratio of different systems, it is important to compare the different systems with regard to system 
reliability and operation and maintenance aspects. Until now, this comparison has been 
problematic due to the following factors: 


Most systems have not been properly studied, or the results have not been published; 
Different studies have evaluated systems with regard to different parameters; 

Different studies used different methods; and 

Different systems have been evaluated under varying conditions (e.g., varying road and 
climate conditions). 


ao Tf 
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Therefore, WTI/MSU evaluated different types of animal detection systems from different 
vendors at the same site and under similar circumstances. Phase 1 of the project involved 
designing and implementing the backbone of the “Roadside Animal Detection Systems” (RADS) 
test-bed in a controlled access environment, followed by the installation of selected animal 
detection systems. During Phase 2, WTI/MSU measured the reliability and the costs and benefits 
of the systems. For the final phase, Phase 3, several sites in Montana were reviewed for possible 
installation of the best performing animal detection system. Finally, the project provided tech 
transfer to transportation agencies, including FHWA and the Montana Department of 
Transportation (MDT), vendors of animal detection systems, the general public, and the 
scientific community. 


1.3. Project Goals and Objectives 


The objectives of this project were to: 


e Develop a high-level concept of operations that includes transportation agencies, the 
traveling public, vendors of animal detection systems, and researchers. 


e Design and implement the RADS test-bed backbone utilizing a systems engineering 
approach. 


e Install selected animal detection systems in the RADS test-bed backbone. 


e Measure and compare the reliability of the different types of animal detection systems 
from different vendors included in the RADS test-bed. 


e Document the experiences with installation and operation and maintenance, including 
system costs. 


e Review the animal detection systems included in the RADS test-bed with regard to 
National ITS Architecture standards. 


e Develop standards for recommended performance requirements of animal detection 
systems. 


e Promote cooperation and communication between transportation agencies and vendors. 


e Provide feedback to vendors to help them build systems that meet national ITS 
architecture standards and recommended performance requirements. 


e Review one or more sites in Montana for possible installation of an animal detection 
system. 


e Provide the study results to the funders, including FHWA and MDT, and advise them on 
future investments in animal detection systems and their potential applications. 


e Provide the study results to other transportation agencies, the scientific community, and 
the general public. 


The goals, objectives, and measures of effectiveness for this project are defined in Table 1.1. The 
report has been organized according to these goals and objectives. The measures of effectiveness 
will allow us to answer the main research questions and tie back into the goals and objectives. 
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Table 1.1: Goals, objectives, and measures of effectiveness. 


Provide a vision to transportation agencies and participating 
vendors of how animal detection systems may work in the 


future, and what criteria should be included in the RADS 


test-bed 


Provide advice to transportation agencies on the selection of 
animal detection systems that are reliable and that minimize 


installation, operation, and maintenance costs 


Help develop animal detection systems that are integrated 
into national ITS architecture and standards, and that meet 
certain minimum performance requirements 


Promote informed decisions on the selection and use of 


animal detection systems that are reliable and that minimize 


installation, operation, and maintenance costs 


Western Transportation Institute 


Develop a high-level concept of operations 


Design and implement RADS test-bed backbone, 
facilitating integration of animal detection systems 


Install selected animal detection systems in the RADS 
test-bed backbone 


Identify the most reliable animal detection systems 


2 


Identify the systems that require the least installation. 
operation, and maintenance efforts 

Review the animal detection systems included in the 
RADS test-bed with regard to national ITS architecture 
and standards 


Develop recommended performance requirements 
Promote cooperation and communication between 
transportation agencies and vendors 


Provide feedback to vendors to help them build systems 
that meet national ITS architecture and standards and 
recommended performance requirements 


Review one or more sites in Montana for possible 
installation of an animal detection system 


Provide the study results to the funders, including 
FHWA and MDT, and advise them on future investments 
in animal detection systems and their potential 
applications 


Provide the study results to other transportation agencies, 
the scientific community, and the general public 


Introduction 


Measures of Effectiveness 
Acceptance by transportation agencies (FHWA and MDT) and 
vendors 


Must meet requirements 
Must meet requirements 


1. The percentage of false detections (false positives) for each 
system 

2. The percentage of missed animal (or model) crossings (false 
negatives) for each system 

3. Downtime of the system (time that the system is not 
operational or time that the system does not function according 
to the specifications of the vendor and the expectation of the 
researchers) 

Document the experiences with installation, operation, and 
maintenance (expressed in terms of time and money, if possible) 
Compare national ITS architecture and standards with that of the 
systems included in the RADS test-bed 


Acceptance by transportation agencies (FHWA and MDT), 
vendors, the general public, and researchers 
Document the lessons learned from public—private partnerships 


Acceptance by vendors leading to modifications to their 
products on the long-term 


One or more sites will be evaluated with respect to a range of 
parameters, including historic road-kill, road characteristics, 
terrain characteristics, vegetation characteristics, etc. 

Acceptance of the final report by the funders (FHWA and MDT) 


Outreach through speaking at selected conferences, the 
publication of peer-reviewed scientific articles in international 
journals, and exposure in popular media (newspapers, radio, 
television) 
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1.4. Project Location 


The RADS test-bed was installed in a controlled access environment near Lewistown, Montana. 
WTI/MSU is using models for wildlife (i.e., domesticated species—horses and llamas) rather 
than wild animals. The reasons for using a controlled access environment and models for wildlife 
are: 


e Acontrolled access environment reduces or eliminates vandalism, theft, and accidents. 
The reduction or elimination of such events minimizes operation and maintenance costs 
as well as liability risks during the project. 


e Acontrolled access environment allows for a better comparison of the reliability of the 
different animal detection systems by reducing or eliminating vandalism, theft, and 
accidents that may confound the results of the study. Although a “real world” setting 
could potentially result in new or additional experiences with operation and maintenance, 
it may not result in a fair comparison of these aspects as not all systems may suffer 
equally from vandalism, theft, or accidents. A controlled access environment is the best 
guarantee for similar circumstances for a comparison of the reliability of the different 
animal detection systems. 


e Acontrolled access environment reduces the number of parties involved in the project. 
This makes the project easier to manage and improves adherence to the schedule. 


e Acontrolled access environment allows the project to direct the majority of the resources 
to the actual comparison of the animal detection systems rather than to incident 
management. 


e By using models for wildlife, rather than wild animals, WTI/MSU can control the 
number and location of “animal movements.” This allows for a shorter test-bed and 
testing period. A shorter test-bed greatly reduces the costs for sensors, poles, other 
equipment, and operation and maintenance. A shorter testing period allows us to 
complete the project in a shorter time frame. 


e Vendors are more likely to donate equipment for the duration of the study if the risk of 
vandalism or theft is reduced, and the number of sensors and other equipment required is 
minimized. 


However, there are some drawbacks to a controlled access environment and working with 
models for wildlife. WTI/MSU has addressed these issues as follows. 


e Acontrolled access environment is not the same as a real roadside environment. 
However, other systems installed in real roadside environments have already given 
insight into problems that may be encountered there (see review in Huijser et al., 2006a). 
Furthermore, a project such as this is already challenging from a technical perspective 
making it wise to eliminate unnecessary risks. The project is primarily aimed at 
comparing different technologies rather than exploring the hazards of a roadside 
environment. 


Western Transportation Institute Page 5 


Reliability of animal detection systems Introduction 


e Models for wildlife may differ from wild animals, so WTI/MSU has selected models 
(llamas and horses) that are relatively similar in size to the “target” species (e.g., deer, 
pronghorn, elk, or moose). 


WTI/MSU installed the RADS test-bed on the grounds of Lewistown airport in Montana. Since 
part of the airport is being transformed into a cold region and rural transportation research, 
maintenance and operations test-bed (“TRANSCEND”), it provides an ideal location for the 
RADS test-bed as this project aims to test technology under a range of weather conditions, 
especially cold weather. Furthermore, animal detection systems typically find their applications 
in rural settings. However, the following challenges had to be addressed: 


e Poles and equipment had to be installed next to a paved road section that one can drive on 
to mimic traffic for potential future research efforts. This causes potential conflicts with 
other uses that may not allow for any obstacles near the paved sections. However, since 
the test-bed is relatively short (91 m (300 ft)), the researchers were able to find a suitable 
location. To reduce safety risks, all poles for the detection equipment were equipped with 
a break-away system. 


e The location had to be relatively close to existing power lines (110 V). Using power from 
the grid rather than solar panels or generators allowed the researchers to focus on the 
comparison of the different technologies, rather than potential challenges with different 
power sources. 


e Because domesticated animals (horses and llamas) were used as wildlife models, an 
enclosure had to be constructed. Arrangements were made for the feeding and care of the 
animals, and permits were obtained to have the animals be part of an experiment. 
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2. CONCEPT OF OPERATIONS 


Author: Marcel P. Huijser, Western Transportation Institute, College of Engineering, Montana 
State University 


2.1. Systems Concepts Studied 


This study deals with animal detection systems based along the road only. It does not deal with 
animal warning systems or vehicle-based detection systems. Animal warning systems detect 
vehicles or trains and then alert large animals through a range of audio and visual signals from 
stations placed in the right-of-way (for details and discussion see Bushman et al., 2001; Huijser 
and McGowen 2003; Hunin 2005; Mulka 2008). Vehicle-based systems (e.g., Bendix, 2002; 
General Motors, 2003; Hirota et al., 2004; Honda, 2004) only inform drivers of the possible 
presence of animals in vehicles equipped with such a detection system. Road-based animal 
detection systems, however, are designed to inform all drivers, regardless of what equipment 
their vehicle may or may not have. 


2.2. Concept of Operation 


A road-based animal detection system consists of two parts: one part detects large animals as 
they approach the road, and the other part warns the drivers after detection has occurred (Figure 
2.1). A transportation agency or natural resource management agency usually takes the initiative 
for site- and species-specific mitigation measures. Site selection is often based on accident 
reports and road mortality data for large animal species. The transportation agency and natural 
resource management agencies then decide on the appropriate mitigation measure, in this case an 
animal detection system. After a detection technology and vendor have been selected, an animal 
detection system is built and delivered by the vendor. An installation contractor then puts the 
system in place. 


Once the system is installed and working according to the agreed-upon specifications, the 
transportation agency may operate and maintain the system. In some cases natural resource 
management agencies may assist with checking up on the system. Currently most systems have 
to be checked at the site regularly to verify that the system is indeed operating correctly. In some 
cases there is remote access to the detection data and system diagnostics through land-based 
phone lines, or cellular or satellite phone. In the future there may be algorithms in place that 
screen the data continuously for unusual patterns that may indicate that there is a problem with 
the system or parts thereof. Once a problem with the system is detected, a person may be notified 
through an automated system. Figure 2.1 shows the concept of operations for animal detection 
systems. Arrows indicate the direction of output and processes. Solid lines indicate output and 
processes that exist already. Dotted lines indicate output and processes that may be developed in 
the future. 
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Figure 2.1: Concept of operations 


The transportation agency provides information to the traveling public about the purpose and the 
location of the animal detection system. This information should be provided just before drivers 
get to the area covered by the animal detection system. Road signs and highway advisory radio 
messages are the most obvious ways to deliver this information to the driver. When approaching 
the animal detection system a driver may be confronted with an activated warning signal 
indicating that a large animal has been detected and is present on or near the road at that time. 


In the future the information about the purpose and the location of the animal detection system 
may also be delivered to an on-board computer inside the vehicle. The information would be 
provided as soon as the vehicle gets within a certain radius of the animal detection system. This 
procedure would require a two-way GPS-based communication system. The warning signal may 
also be delivered to an on-board computer inside the vehicle. 


2.3. System Reliability and Effectiveness 


In order to reduce the number of animal—vehicle collisions, animal detection systems need to 
detect animals reliably, and they also need to influence driver behavior so that drivers can avoid 
a collision. 


Most animal detection system technologies are vulnerable to “false negatives” and “false 
positives.” False negatives occur if an animal approaches but the system fails to detect it. False 
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positives occur if the system reports the presence of an animal, but there is no animal present. 
Numerous false positives may result in drivers regarding the system no differently than a 
permanently flashing warning light not connected to sensors, thus failing to convey the warning 
for acute danger. False negatives should be avoided or kept to an absolute minimum, as drivers 
expect an animal detection system to detect all or nearly all large animals that approach the road. 
False positives should also be minimized, but it is probably more acceptable to have a few false 
positives than a few false negatives. Nevertheless, it is important that animal detection systems 
are reliable, as drivers are expected to respond to the warning signals. 


Once an animal detection system reliably detects the target species and the warning signals and 
signs are activated, driver response determines how effective the system ultimately is in avoiding 
or reducing animal—vehicle collisions. Figure 2.2 splits driver response into two components: 
increased driver alertness and lower vehicle speed. 


Reliable warning signals 


Increased driver awareness; large 
animals may be on or near the road 


—— 


| Lower vehicle speed | Increased driver alertness 


Reduced reaction time 
when confronted with a 
large animal ahead 


Shorter 


stopping 
distance 


| Vehicle may not 
hit animal 


Vehicle may hit 
animal at lower speed 


Figure 2.2: Warning signals and driver response 


A higher state of alertness of the driver, lower vehicle speed, or a combination of the two can 
result in a reduced risk of a collision with the large animal or less severe collisions. A reduced 
collision risk and less severe collisions mean fewer human deaths and injuries and less property 
damage. In addition, fewer large animals are killed or injured on the road without having been 
restricted in their movements across the landscape and the road. Furthermore, fewer large dead 
animals will have to be removed, transported, and disposed of by road maintenance crews. 
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3. TEST-BED DESIGN, DETECTION SYSTEMS, AND TEST ANIMALS 


Authors: Marcel P. Hutjser, Tiffany D. Holland, Matt Blank & Shaowei Wang, Western 
Transportation Institute, College of Engineering, Montana State University 


3.1. Test-Bed Location and Design 


The RADS test-bed is part of the TRANSCEND cold region rural transportation research facility 
and is located along a former runway at the Lewistown Airport in central Montana (Figure 3.1). 
The test-bed location experiences a wide range of temperatures, and precipitation ranges include 
mist, heavy rain, and snow; the topography is flat, and the rocky soil does not sustain much 
vegetation that may obstruct the signals transmitted or received by the sensors. The test-bed 
consists of an animal enclosure, nine different animal detection systems, and six infrared 
cameras with continuous recording capabilities (Figures 3.2 through 3.5). The distance covered 
by the systems (except for System 9) was 91 m (300 ft) (from the left to the right side of the 
enclosure). 


Figure 3.1: The location of the test-bed along a former runway at the Lewistown Airport in 
central Montana. The current municipal airport is located on the upper right of the photo. 
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Figure 3.2: Test-bed design including an animal enclosure, the nine detection systems (open 
circles represent the sensors), the six infrared (IR) cameras aimed at the enclosure from the side 
(solid circles), and the office with data recording equipment. The arrows show the direction 
towards which each sensor or transmitter is pointed. 


Figure 3.3: The test bed with the remote office, poles with animal detection systems attached to 
them, the shelter, and a llama (Photo: Marcel Huser, WTI/MSU). 
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Figure 3.4: Some of the sensors of the animal detection systems (Photo: Marcel Huijser, 
WTI/MSU). 


Figure 3.5: The infrared cameras that monitor animal movements in the enclosure (Photo: Marcel 
Huser, WTI/MSU). 
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The animal enclosure includes shelter, water, and several areas alongside the fence designated 
for feeding. These three resources are located in different parts of the enclosure to maximize 
animal movement through the detection areas. 


3.2. Animal Detection Systems 


During the first five tests, which were conducted from January through May 2007, there were 
eight systems, all installed parallel to each other (Table 3.1). Five of these were area-cover 
systems and the other three systems were break-the-beam systems (Table 3.1). A second STS 
break-the-beam system was installed on July 19, 2007, resulting in a total of nine systems. Two 
of the systems required two detectors to cover the 91 m (300 ft) distance. One of these systems 
(System 8, Xtralis 1-2) had its two sensors installed on a pole in the middle of the 91 m (300 ft) 
distance, with the sensors facing opposite directions (Figure 3.2). The other system (System 2, 
Xtralis 5-6) had a detector installed at each end with the sensors facing each other (Figure 3.2). 
In addition, there was one system that did not cover the 91 m (300 ft) and for which only one set 
of sensors was available (System 9, Goodson). This system was installed across a shorter section, 
equivalent to the maximum distance for this particular system 27 m (90 ft) (Figure 3.2). 
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Table 3.1: The characteristics of the nine animal detection systems. See appendix A for 
manufacturer contact details. 


System Manufacturer ID System Signal type Maximum range Installation date 
number and system number type 
: name 
(Figure 
3.2) 
1 Xtralis 7 Area cover | Passive IR 500 ft (152 m) Sep 21, 2006 
(ADPRO) 
2 Xtralis 5-6 Area cover | Passive IR 200 ft (61 m) (one | Sep 21, 2006 
detector on each 
(ADPRO) side) 
3 STS 1 Break-the- | Microwave radio | % mi (402 m) Oct 19, 2006 
(RADS 1) beam (+ 35.5 GHz) 
4 STS 2 Break-the- | Microwave radio | Well over % mi Jul 19, 2007 
(RADS II) beam (+ 35.5 GHz) (402 m) 
5 Calonder 1 Break-the- | Laser 984 (built-up areas) | Sep 21-22, 2006 
Energy (CAL beam —1148 ft (open 
92, LS-WS- areas) (300-350 m) 
WE 45) 
6 Calonder 2 Area cover | Passive IR 328 ft (100 m) Sep 21-22, 2006 
Energy (CAL 
92, IR-204- 
319/M3) 
7 Camrix Areacover | IR ITS Camera | 300 ft (91 m) Oct 19-31, 2006 
(A.L.E.R.T.) Technology (Note: | unit 
detects both sides 
of a road) 
8 Xtralis 1-2 Area cover | Passive IR 200 ft (61 m) (2 Aug 8, 2006 
detectors, one 
(ADERO) facing each way) 
9 Goodson Break-the- | Active IR 90 ft (27 m) Dec 2006 
beam 


The six infrared cameras (Fuhrman Diversified, Inc.) were installed perpendicular to the 
detection systems on November 8-9, 2006. These cameras and a video recording system record 
all animal movements within the enclosure continuously, day and night. The animal detection 
systems saved their individual detection data with a date and time stamp. These data were 
compared to the images from the infrared cameras, which also had a date and time stamp, to 
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investigate the reliability of each system. Cones within the enclosure defined the detection zone 
for each system (Figure 3.6). 


Figure 3.6: The detection zones and detection lines were marked with cones to be able to record 
the position of the animals (Photo: Marcel Huijser, WTI/MSU). 


Area-cover systems are designed to detect animals within a certain area and range from a sensor. 
This area is typically cone-shaped—narrow close to the sensor and wider as the distance from 
the sensor increases (Figure 3.7). All area-cover systems tested in this study detect animals based 
on body heat and motion. Break-the-beam systems consist of a transmitter that transmits a signal 
to a receiver. Break-the-beam systems detect animals when their body blocks the signal or when 
the signal received by the receiver is greatly reduced. The break-the-beam systems tested in this 
study use infrared, laser or microwave radio signals. 


The detection area is the area within which area-cover systems should detect large animals, and 
the detection line is the line between sensors where break-the-beam systems should detect large 
animals (Figure 3.7). The detection areas and detection lines were indicated by the manufacturers 
and were marked with cones that were visible on the images from the individual cameras. Area- 
cover systems have relatively large, cone-shaped detection areas, whereas break-the-beam 
systems have a detection line that is linear or mostly linear in shape, although the STS 1 system 
that used microwave signals had a 3° angle from the transmitter, which resulted in a detection 
area that was 2.4 m (7.8 ft) wide at 91.4 m (300 ft) from the transmitter (Pers. com., Lloyd 
Salsman, Sensor Technologies & Systems, October 10, 2007). 
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Figure 3.7: Schematic representation of break-the-beam and area-cover systems showing the 
detection line (or center line) for break-the-beam and area-cover systems, and the detection area 
for area-cover systems. 


The detection technology of the different systems is described below. Photos of the systems are 
in Figures 3.8 through 3.14). 


The Xtralis systems detect changes in infrared radiation (8—13um) (Pers. com., Andreas 
Hartmann, Xtralis, October 1, 2007), which allows the system to detect the motion of an object 
against a stationary background. Such motion leads to changes in infrared radiation, which are 
processed by the system. Filtering and algorithms help distinguish between large animals and 
other objects to help reduce or prevent false detections. The STS RADS systems transmit 
microwave radio signals (around 35.5 GHz) (Huijser et al., 2006b; Salsman & Wilson, 2006). 
These signals are received by a sensor on the other end, and whenever an animal or object passes 
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between the sensors, the signal is reduced. If certain thresholds are met, the reduction in signal 
strength results in a detection. RADS II is more compact than RADS I and has parts integrated 
into fewer components. The detection line of the STS 1 system is about 2.4 m (7.8 ft) wide at 
91.4 m (300 ft) from the transmitter (Pers. com. Lloyd Salsman, Sensor Technologies & 
Systems, October 10, 2007). For the STS 2 system the detection line is 40.6 cm (16 in) wide 
consistently (Pers. com. Lloyd Salsman, Sensor Technologies & Systems, October 10, 2007). In 
addition, both the STS 1 and STS 2 systems have a wider detection area 4.5 m (15 ft) close to the 
sensors (Pers. com., Lloyd Salsman, Sensor Technologies & Systems, October 10, 2007). 
Calonder Energy 1 transmits a laser signal that is received by a sensor on the other end. 
Whenever an animal or object blocks the laser signal, the system reports a detection. Calonder 
Energy 1 was installed at 105 cm (41.34 in) above the ground. Calonder Energy 2 detects 
changes in infrared radiation as a result of objects moving 0.2—5 m/s (8 in/s — 16.4 ft/s) (Pers. 
com., Giacomo Calonder, Calonder Energy, September 22, 2006; Calonder Energy, not dated). 
Algorithms help distinguish between large animals and other objects to help reduce or prevent 
false detections. This system was installed 3 m (9.8 ft) above the ground, pointing downwards at 
a 3—5° angle. There is a blind spot of approximately 10-12 m (32.8-39.4 ft) directly under the 
sensor, and the detection area is about 3 m (9.8 ft) wide at 100 m (328 ft) from the sensor (Pers. 
com., Giacomo Calonder, Calonder Energy, October 10, 2007). This blind spot is normally 
covered by another passive infrared sensor with a range of 18 m (59.1 ft) (Pers. com. Giacomo 
Calonder, Calonder Energy, October 10, 2007). The Calonder Energy 2 system (IR-204-319/M3) 
was discontinued in 2007 and Calonder Energy now offers an ADPRO unit from Xtralis (Pers. 
com., Giacomo Calonder, Calonder Energy, October 9, 2007). The Animal Location Evasive 
Response Technology (A.L.E.R.T.) system from Camrix uses a camera, optics, infrared 
illumination, and a computer to gather and analyze digital imagery (Pers. com., Mike Doyle, 
Camrix, October 3, 2007). Advanced proprietary machine vision algorithms process the images 
and decide whether a detection should be declared. The Goodson system (TM 1550) transmits an 
infrared signal that is received by a sensor on the other end. Whenever an animal or object blocks 
the infrared signal, the system reports a detection. 
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Figure 3.8: System 1 (Xtralis 7) and System 2 (Xtralis 5-6) mounted on the same pole (Photo: 
Matt Blank, WTI/MSU). 
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Figure 3.9: System 3 (STS I; black tube behind pole) and System 4 (STS II; white sensor in front 
of pole) mounted on the same pole (Photo: Tiffany Holland, WTI/MSU). 
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Figure 3.11: System 6 (Calonder Energy 2) (Photo: Marcel Huijser, WTI/MSU). 
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Figure 3.13: System 8 (Xtralis 1-2) (Photo: Marcel Huijser, WTI/MSU). 
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Figure 3.14: System 9 (Goodson) (Photo: Marcel Huijser, WTI/MSU). 


3.3. System Costs 


The costs for the systems are summarized in Table 3.2. The costs are for one sensor (area-cover 
systems) or one sensor pair (break-the-beam systems) and associated electronics. The costs are 
indicate and based on quotes in the years indicated. Animal detection systems can be combined 
with other mitigation measures such as wildlife fencing (see conceptual drawings in Huijser et 
al., 2006). Dependent on the road length that needs to be covered by a system and the range of 
the sensors, one may calculate a price for two units (to cover both sides of a road) or a price per 
meter road length. 


Note: ICx Radar Systems (formerly STS) has developed a new generation system (RADS II) 
which is quoted for $11,616 (2009) (one transmitter and one receiver, including associated 
electronics, excluding installation, warning signs) (Pers. com. Dan Bjerk, ICx Radar Systems, 
March 18, 2009). 


Note: Goodson (Trailmaster) indicated that a unit that would be customized for animal detection 
along roadways is likely to cost around $1,200 (2009) (one transmitter and one receiver, 
including associated electronics, excluding installation, warning signs). This customized unit 
would have a range of about 183-305 m (600-1,000 ft) and could be powered through solar 
panels or connected to the grid. The unit would have the ability to be daisy chained with other 
units to cover longer road sections (Pers. com. Bill Goodson, Trailmaster, February 26, 2009) 
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Table 3.2: The characteristics of the nine animal detection systems. See appendix A for 
manufacturer contact details. 


System | Manu- ID Indicative | Included/excluded Maximum 
number | facturer number | equipment range 
(Figur and costs 
sure system (USS) (yr 
3.2) 
name of quote) 
1 Xtralis 7 $1,101 Incl.: 1 sensor and (wall) mounting. Excl.: | 500 ft (152 
(ADPRO) (2009) warning signs, installation materials (foundations, | m) 
poles) and installation labor. 
2 Xtralis 5-6 $844 Incl.: 1 sensor (range 61 m) and (wall) mounting. | 200 ft (61 m) 
ADPRO (2009) Excl.: warning signs, installation materials | (one detector 
( ) (foundations, poles) and installation labor. on each side) 
3 STS 1 $13,136 Incl.: 1 transmitter and 1 receiver and associated | '4 mi (402 m) 
RADSI (2005) electronics. Excl: warning signs, installation 
( ) materials (foundations, poles) and _ installation 
labor. 
4 STS 2 $17,314 Incl.: 1 transmitter and 1 receiver and associated | Well over 4 
RADS II (2006) electronics. Excl: warning signs, installation | mi (402 m) 
( ) materials (foundations, poles) and _ installation 
labor. 
5 Calonder | 1 $12,737 Incl.: 1 transmitter and | receiver and associated | 984 (built-up 
Energy (2006) electronics. Excl: warning signs, installation | areas) —1148 
(CAL 92, materials (foundations, poles) and installation | ft (open 
LS-WS- labor. areas) (300- 
WE 45) $9,000 Incl.: 1 laser unit, wiring, control box, and pole. | 350 m) 
(2009) Excl.: installation. 
6 Calonder | 2 $8,000 Incl.: 1 sensor unit, wiring, control box, and pole. | 328 ft (100 
Energy (2009) Excl: installation. m) 
(CAL 92, 
IR-204- 
319/M3) 
7 Camrix $14,400 Incl: 1 camera, computer, enclosure, solar panel, | 300 ft (91 m) 
(A.L.E.R and battery. Excl: installation hardware (pole, 
T.) associated hardware) signage, labor. 
8 Xtralis 1-2 $844 Incl.: 1 sensor (range 61 m) and (wall) mounting. | 200 ft (61 m) 
ADPRO (2009) Excl.: warning signs, installation materials | (2 detectors, 
( ) (foundations, poles) and installation labor. one facing 
each way) 
9 Goodson $260 Incl.: 1 transmitter and 1 receiver and associated | 90 ft (27 m) 
(2009) electronics. Excl: warning signs, installation 
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3.4. Wildlife Target Species and Models 


In a North American setting, animal detection systems are typically designed to detect white- 
tailed deer (Odocoileus virginianus) and/or mule deer (Odocoileus hemionus), pronghorn 
(Antilocapra americana), elk (Cervus elaphus) or moose (Alces alces). For this study, which 
took place within an enclosure, two horses and two llamas were used as models for these wildlife 
target species. Horses are similar in body shape and size to moose, whereas the body shape and 
size of llamas is similar to deer (Tables 3.3 and 3.4). The body size and weight of the individual 
horses and llamas used in this experiment are shown in Table 3.5. 


Table 3.3: Height and length of wildlife target species and horses and llamas. 


Species Height at shoulder Length (nose to tip of tail) Source 
Target species 
Moose 195-225 cm (6'5"-7'5") 206-279 cm (6'9"-9'2") Whitaker (1997) 
Elk 137-150 cm (4'6"-5') 203-297 cm (6'8"-9'9") Whitaker (1997) 
White-tailed deer | 68-114 cm (2'3"-3'9") 188-213 cm (6'2"-7') Whitaker (1997) 
Mule deer 90-105 cm (3'-3'5") 116-199 cm (3'10"-7'6") Whitaker (1997) 
Pronghorn 89-104 cm (2'11"-3'5") 125-145 cm (4'1"-4'-9") Whitaker (1997) 
Models 
Feral horse 142-152 cm (4'8"-5') Whitaker (1997) 
Quarter horse 150-163 cm (4'11"-5'4"") UHS (2007), Wikipedia (2007) 
Llama 91-119 cm (3'-3'11") Llamapaedia (2007) 
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Table 3.4: Body weight of wildlife target species and horses and llamas. 


Species Weight male Weight female Source 
Target species 
Moose 400-635 kg (900-1400 Ibs) | 315-500 kg (700-1,100 Ibs) Whitaker (1997) 
Elk 272-494 kg (600-1089 Ibs) | 204-295 kg (450-650 Ibs) Whitaker (1997) 
White-tailed deer | 68-141 kg (150-310 lbs) 41-96 kg (90-211 Ibs) Whitaker (1997) 
Mule deer 50-215 kg (110-475 lbs) 32-73 kg (70-160 Ibs) Whitaker (1997) 
Pronghorn 41-64 kg (90-140 Ibs) 34-48 kg (75-105 Ibs) Whitaker (1997) 
Models 
Feral horse 360-390 kg (795-860 Ibs) | 270-340 kg (595-750 Ibs) Whitaker (1997) 
Quarter horse 386-540 kg (850-1200 Ibs) UHS (2007), Wikepedia (2007) 
Llama 113-204 kg (250-450 lbs) Llamapaedia (2007) 


Table 3.5: Body size and weight of the horses and llamas used in the experiment (Pers. com., 
Lethia Olson, livestock supplier). 


Individual Height at shoulder Weight 
Horse 1 152 cm (5’) 513 kg (1,130 lbs) 
Horse 2 160 cm (5’3’’) 658 kg (1,450 lbs) 
Llama 1 104 cm (3’5’’) 168 kg (370 Ibs) 
Llama 2 110 cm (3’7%’’) 213 kg (470 lbs) 


3.5. Research 


The research at the TRANSCEND test facility focused on two main questions, which are 
addressed in subsequent chapters: 


e How reliable are the animal detection systems in detecting large mammals (horses and 
llamas) (Chapter 4)? 


e Is the reliability of the animal detection systems influenced by environmental conditions 
(Chapter 5)? 
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4. RELIABILITY TESTS 


Authors: Marcel P. Hutjser, Tiffany D. Holland, Matt Blank & Shaowei Wang, Western 
Transportation Institute, College of Engineering, Montana State University 


4.1. Test Periods, Data Selection, and Data Storage 


Reliability tests 


There were eight test periods with test animals between January 10, 2007 and December 9, 2007 
(Table 4.1). Each test period with animals lasted 7-11 days. 


Table 4.1: Test periods with animals present. 


Test period Start End 

1 January 10, 2007 January 17, 2007 
2 February 19, 2007 February 28, 2007 
3 March 16, 2007 March 25, 2007 
4 April 22, 2007 May 2, 2007 

2 May 24, 2007 June 3, 2007 

6 July 20, 2007 July 30, 2007 

fi August 23, 2007 September 3, 2007 
8 November 30, 2007 December 9, 2007 


Camera images were recorded on site on a hard drive that is capable of storing 10—14 days of 
data. Camera images from three types of time periods were reviewed and compared to the 


detection logs of the individual systems to measure the reliability of each system: 


e Stratified random with animals present: Three, one-hour-long sections of video were 
randomly selected for each test day for review. 


e Non-random with animals present: These time periods were chosen based on the 
occurrence of unusual detection patterns (i.e., times with an unusually high or low 
number of detections) and certain weather events. Some of these weather events included 
the occurrence of precipitation (i.e., rain, snow or sleet), extreme temperatures (hot or 
cold), and low visibility (i.e., fog, heavy rain or snow). The length of time reviewed 
depended upon the length of the event itself; the time period analyzed varied in length. 


e Non-random without animals present: After each test with horses and llamas present, 
images were recorded for an additional 12-14 days (during these days there were no 
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Reliability tests 


domesticated animals present). This is the maximum number of days that could be stored 
on the hard drive. Additional non-random time periods were selected from these days. 
These time periods were also chosen based on unusual detection patterns and certain 


weather conditions. 


The images that were analyzed were all saved on DVD. Time periods that were not analyzed 
were not saved. The number of hours from which images were analyzed for each system is listed 
in Table 4.2. 


Table 4.2: The number of hours from which images were analyzed for each system. 


System Manufacturer and ID Stratified Non-random Non-random Total 
number system name number | random with with animals without (hours) 
: animals present animals 
(Figure 
3.2) present (hours) present 
; (hours) (hours) 
1 Xtralis 7 225.0 16.9 28.0 269.9 
(ADPRO) 
2 Xtralis 5-6 225.0 16.9 28.0 269.9 
(ADPRO) 
3 STS 1 225.0 16.9 28.0 269.9 
(RADS I) 
4 STS 2 91.0 Bee) 20.2 116.7 
(RADS II) 
5 Calonder Energy (CAL 1 225.0 16.9 28.0 269.9 
92, LS-WS-WE 45) 
6 Calonder Energy (CAL 2 225.0 16.9 28.0 269.9 
92, IR-204-319/M3) 
ah Camrix 225.0 16.9 28.0 269.9 
(A.L.E.R.T.) 
8 Xtralis 1-2 225.0 16.9 28.0 269.9 
(ADPRO) 
9 Goodson 225.0 16.9 28.0 269.9 


A modification to the Camrix system towards the end of the field tests (29 November 2007, see 
Table 5.2, Chapter 5) resulted in a significant change in the reliability performance of the system 
(see Table 5.4, Chapter 5). Therefore, the reliability parameters for the Camrix system presented 
in this current chapter are not only presented as totals for the entire study period, but they are 
also presented separately before and after system modification took place. The number of hours 
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from which images were analyzed for the Camrix system before and after system modification 


are: 


Stratified random with animals present: before: 197.0; after: 28.0. 
Non-random with animals present: before: 13.4; after: 3.5. 


Non-random without animals present: before: 15.5; after: 12.5 (8.0 with detection data 
available and 4.5 with no detection data available). 


Total: before: 225.9; after: 44.0 (39.5 with detection data available and 4.5 with no 
detection data available). 


4.2. Video Review and Reliability Parameters 


The time periods reviewed were analyzed for valid detections, false positives, false negatives, 
intrusions in the detection area, and downtime. These terms are defined below. 


Valid detections — A valid detection was defined as “the presence of an animal within the 
detection zone of the system in conjunction with a corresponding detection recorded by 
the system’s data logger.” The number of valid detections depends on the frequency with 
which a system “scans” for the presence of an animal. Sensors that sense an area typically 
sense continuously and have, by definition, more detections than break-the-beam systems 
that only sense the presence of an animal when it crosses the detection line (which is a 
much smaller “area”). Some break-the-beam systems also have a “reset” period after a 
detection before the system can report the next detection. For the time periods reviewed, 
the date, time, and species were recorded for all valid detections. 


Cases in which humans, birds, dogs, or other non-target species entered the enclosure 
were not considered in evaluating valid detections. However, in the rare circumstance 
when a deer happened to enter the enclosure, the incident was included in the analysis, 
and it was evaluated for valid detections in the same manner as described above. 


False positives — A false positive was defined as “when the system reported the presence 
of an animal, but there was no animal in the detection zone.” Thus, each incident in 
which a system’s data logger recorded a detection, but there was no animal present in the 
detection zone of that system, was recorded as a false positive. The date and time were 
recorded for all false positives. 


Cases in which humans, birds, dogs, or other non-target species entered the enclosure 
were considered a valid explanation for a detection and were not recorded as a false 
positive. 


False negatives — A false negative was defined as “when an animal was present but was 
not detected by the system.” However, due to animal behavior and the design of some 
detection systems (i.e., some systems are desensitized by the continuous presence of an 
animal), there are several ways for a false negative to occur. Therefore, various types of 
false negatives were distinguished and these were recorded separately. The date, time, 
and species were recorded for each type of false negative. 
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The simplest type of false negative, recorded as “false negative,” occurred when an 
animal completely passed through “the line of detection” without lingering but was not 
detected by the system. The line of detection was defined by the straight line between 
either end of the system (in the case of break-the-beam systems, or area-cover systems 
that are relatively narrow— up to 4.6 m (15 ft) or a line through the center (or close to the 
center) of a wide detection area (wider than 4.6 m (15 ft); System 7 (Camrix A.L.E.R.T.)) 
(see Figure 3.7). If an animal lingered in the detection zone but did not completely cross 
the line of detection or centerline, it was not deemed a false negative. After a valid 
detection at least three minutes had to pass before another animal movement across the 
centerline could be viewed as a false negative. However, if two or more animals passed 
the centerline within three minutes of each other, and if they were all detected, all 
passages were considered a valid detection across the centerline. The three minute “reset” 
period was put in effect because: 


O Some sensors are desensitized after a detection and need some time before they 
can detect another animal. For example, the vendor of the STS 1 system 
recommends three minutes reset time for the sensors to become fully sensitive 
again after a detection. 


Oo The warning signs of an animal detection system need to stay activated for a 
certain amount of time after a detection anyway. Therefore it is not essential to 
have an animal detection system detect multiple animals within a short time. 
Based on an analysis of patterns in the detection data from a field site it was 
concluded that it seemed appropriate to have warning signs be activated for three 
minutes after a detection had occurred. The three minute time period was found to 
be an appropriate balance between warning the drivers for animals that may still 
linger on or close to the road and not exposing drivers to unnecessary warnings. 


Another type of false negative, recorded as “false negative 1,” occurred when an animal 
lingered in the detection zone before completely passing through the line of detection 
without a detection by the system. If the system did not detect the animal as it completely 
passed through the line of detection, and if it was three minutes or longer since the 
system last detected an animal, it was considered a false negative. If the system did not 
detect the animal as it completely passed through the line of detection, and it was less 
than three minutes since the system last detected an animal, it was considered neither a 
false negative nor a valid detection. 


A third type of false negative, recorded as “false negative 2,” occurred when one animal 
lingered in the detection zone without a detection by the system, while a second animal 
(or multiple animals) completely passed through the line of detection. If the system did 
not detect the second animal as it completely passed through the line of detection, and it 
was three minutes or longer since the system last detected an animal, it was considered a 
false negative. If the system did not detect the animal as it completely passed through the 
line of detection, and it was less than three minutes since the system last detected an 
animal, it was considered neither a false negative nor a valid detection. 


In addition to valid detections, false positives and false negatives, the total number of times an 
animal should have been detected was recorded. The number of times an animal should have 
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been detected was the sum of the number of times an animal crossed the line of detection and 
was detected and the total number of false negatives, regardless of the type of false negative. 


Cases in which humans, birds, dogs, or other non-target species entered the enclosure were not 
considered in evaluating false negatives. However, when deer entered the enclosure, the incident 
was included in the analysis. 


e Intrusions in detection area — An intrusion was defined as “the presence of one or 
multiple animals in the detection zone.” An intrusion began when one or more animals 
entered the detection zone and ended when all animals left the detection zone. Each 
intrusion resulted in one of the three event types described below. The event types were 
hierarchical—while an intrusion was in progress, the classification could change from E2 
to El, from E3 to E2, or from E3 to El, but not from E1 to E2 or E3 or from E2 to E3. 


The first type of event, classified as “event 1” or “El,” occurred when an animal entered 
the detection zone and was detected by the system. 


The second type of event, classified as “event 2” or “E2,” occurred when an animal 
completely crossed the line of detection but was not detected by the system. After each 
valid detection, there was a reset time of three minutes before evaluating that system for 
an event 2. 


The third type of event, classified as “event 3” or “E3,” occurred when an animal entered 
the detection zone but did not fully pass through the line of detection and was not 
detected by the system. After each valid detection, there was a reset time of three minutes 
before evaluating that system for an event 3. If an animal entered the detection zone 
within three minutes of the last valid detection and if it remained in the detection zone 
after the three-minute reset time had expired, it was evaluated for an event 3. Date, time, 
and species were recorded for each event 3. 


Event 3 was only analyzed with regard to System 7 (Camrix A.L.E.R.T.). This was the 
only system that had a large enough detection area on either side of the detection line to 
accurately analyze for an event 3. For all other systems, if the animal was at the 
centerline or in the narrow detection zone, and it did not fully cross the centerline (e.g., it 
turned back), and the animal was not detected, it was not an event. In addition, event 3 
applied only to System 7 (Camrix A.L.E.R.T.) because all other area sensors that were 
tested acted more like break-the-beam systems (i.e., they were more sensitive to 
movement near the centerline), whereas System 7 (Camrix A.L.E.R.T.) was equally 
sensitive to movement in all areas of the detection zone. 


e Downtime — Downtime was defined as “the time when the system was not working at all 
or when it was not working according to the expectations of the researchers or the 
specifications of the vendor.” Date, time, and duration of downtime were recorded for 
each system. 


4.3. Data Analyses 


Since this chapter focuses on the reliability of the animal detection systems, only the stratified 
random time periods were selected for the data analyses. Non-randomly selected time periods 
would result in a bias for one or more systems because the non-random periods were selected, in 
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part, because of unusual detection patterns for one or more systems. The data were combined for 
all test periods. However, time periods that were classified as downtime or time periods for 
which no detection data were available due to external circumstances (e.g., power outage) were 
excluded from the analyses. 


The following parameters were calculated for each system: 
e The average number of valid detections per hour: 
— N 


= t (valid detections) 
N tvalia det ections) 


h(with data available) 
Where: 


N , (valid detections) = total number of valid detections 


Nj, (with data available) total number of hours for which detection data were available 


e The percentage of false positives for each system: 


Ft F; 


Fr= *100 = 


t (detections recorded by system) t (valid detections) 


*100 
N 


+Fy 
Where: 
F, = total number of false positives 


N = total number of detections recorded by a system 


t (detections recorded by system) 


N , (valid detections) = total number of valid detections 


e The average number of false positives per hour for each system: 


=. FS 


N h(with data available) 
Where: 


F, = total number of false positives 


N, iidemnie = total number of hours for which detection data were available 
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e The percentage of false negatives for each system: 


: - 
F- =" #199 = N___] 0 
N 


t (center line) d (center line) 


Where: 


F, = total number of false negatives (false negatives, false neg. 1, and false neg. 2) 


N (centerline) = total number of times an animal crossed the centerline and should have been 
detected 
N = total number of times an animal crossed the centerline and was detected 


d (center line) 


Note that the percentage was calculated for false negatives, false negatives 1, and false 
negatives 2 individually. Since the total number of false negatives varied between these 
categories, the sum of the percentages for false negatives, false negatives 1, and false 
negatives 2 do not equal the percentage of the total number of false negatives. 


e The average number of false negatives per hour for each system: 
— 
N 


h(with data available) 
Where: 


F, = total number of false negatives 


N = total number of hours for which detection data were available 


h (with data available) 


Note that the percentage of false negatives was also calculated for false negatives, false 
negatives 1, and false negatives 2 individually. 


e The percentage of intrusions detected for each system (i.e., animal presence in the 
detection area): 
~ fa «199 =F _ 


% det ected 
I t 1 = 2 


*100 


For System 7 (Camrix) the percentage of intrusions detected for each system was also 
calculated in a different way. This second calculation was possible because the detection 
area for system 7 was large enough for the researchers to distinguish between different 
types of intrusions. The detection area for the other systems was linear in shape and did 
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not allow the researchers to distinguish between certain types of intrusions. The 
alternative way to calculate intrusions for system 7 (Camrix) was: 


I 


I, E 
_ (Camrix) a 1 ¥e 
%detected(Camrix) ~ , 100 = ———————__ * 100 


Eek ASE, 


t(Camrix) 
Where: 


I ,= total number of intrusions detected 
I,= total number of intrusions 


E, = total number of event 1 
E> = total number of event 2 


E3 = total number of event 3 


The values for the reliability parameters can be compared between the individual systems (this 
chapter) and recommended norms for system reliability (chapter 8). However, it is important to 
note that, although the test conditions were standardized as much as possible, the reliability 
values for the individual systems were not measured under the exact same conditions: 


e The number of hours on which the reliability parameters are based was not the same for 
each system as the systems were not all installed at the same time, and hours classified as 
downtime or hours for which no data were available were excluded from the analyses. 


e Even when the number of hours was the same for some of the systems and when the 
observations related to the same hours, the animals did not trigger the different systems at 
the same time. Therefore the conditions under which the reliability of the individual 
systems were evaluated were similar, but not exactly the same. 


e Even when the number of hours was the same for some of the systems and when the 
observations relate to the same hours, the number of times that an animal was present in 
the detection area or the number of times that an animal crosses the detection line and 
thus should or could have been detected was different for each system. 


e Even when the number of hours was the same for some of the systems and even when the 
observations related to the same hours, some technologies resulted in multiple detections 
as an animal entered the detection area or crossed the detection line, whereas other 
technologies would have one detection for an animal that passed through the line of 
detection. 


Therefore, some caution should be used when comparing the values of the reliability parameters 
between individual systems or to recommended norms for system reliability. 


4.4. Results 


The number of hours that detection data were available for analyses for each system is shown in 
Table 4.3. The data in the table show that the reliability parameters calculated for STS II and 
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Calonder Energy 2 were based on relatively few hours compared to the other systems, indicating 
that the reliability parameters for these two systems may be less robust than for the other 
systems. Furthermore, Calonder Energy 2 had relatively much downtime compared to the other 
systems, STS I and Camrix had some downtime, and the other systems had no downtime. The 
number of hours for which no detection data were available due to external circumstances (e.g., 
power outage) is also shown in Table 4.3. 
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Table 4.3: The number of hours that detection data were available for analyses for each system. 


System | Manufacturer and system ID Downtime No detection data | Detection data Total 
number name number (hours) available (hours) available for (hours) 
(icure analyses 
3.2) (hours) 
1 Xtralis 7 0 46 179 225 
(ADPRO) 
2 Xtralis 5-6 0 46 179 225 
(ADPRO) 
3 STS 1 28 81 116 225 
(RADS I) 
4 STS 2 0 34 57 91 
(RADS II) 
5 Calonder Energy (CAL 1 0 76 149 225 
92, LS-WS-WE 45) 
6 Calonder Energy (CAL 2 134 63 28 225 
92, IR-204-3 19/M3) 
7 Camrix 18 0 207 225 
(A.L.E.R.T.) 
8 Xtralis 1-2 0 46 179 225 
(ADPRO) 
9 Goodson 0 0 225 225 
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Reliability tests 


Figure 4.1 shows the average number of valid detections per hour for each system. Camrix had 
much higher numbers of valid detections compared to the other systems. The Xtralis systems 
also had relatively high numbers of valid detections. The other systems (STS I, STS II, Calonder 
Energy 1, Calonder Energy 2, and Goodson) had relatively low numbers of valid detections. 
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Figure 4.1: The average number of valid detections per hour for each system (Camrix: before 
(157.28) and after (72.79) system modification). 


The percentage of false positives was relatively low for all systems. Xtralis 1-2, Goodson, 
Calonder Energy 1, and Camrix had some false positives while the other systems had no false 


positives. 
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Figure 4.2: The percentage of false positives for each system (Camrix: before (0.07) and after 


(0.00) system modification). 
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The average number of false positives per hour was relatively low for all systems (Figure 4.3). 
Xtralis 1-2, Goodson, Calonder Energy 1, and Camrix all had fewer than 0.1 false positives per 
hour while the other systems had no false positives. 
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Figure 4.3: The average number of false positives per hour for each system (Camrix: before 
(0.11) and after (0.00) system modification). 


The percentage of false negatives was lowest for Calonder Energy 1, Calonder Energy 2, and 
Goodson (Figure 4.4).The percentage of false negatives was highest for STS I and Camrix. 
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Figure 4.4: The percentage of false negatives for each system (Camrix: before (30.41; 7.00; 
8.44; 21.22 respectively) and after (27.00; 20.59; 0.00; 10.00 respectively) system 
modification). 
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The average number of false negatives per hour was lowest for Calonder Energy 1, Calonder 
Energy 2, and Goodson (Figure 4.5). The highest values occurred for STS I and Xtralis 1-2. 
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Figure 4.5: The average number of false negatives per hour for each system (Camrix: before 
(1.01; 0.17; 0.21; 0.62 respectively) and after (0.36; 0.25; 0.00; 0.11 respectively) system 
modification). 


The following systems detected all or nearly all intrusions (i.e., animal movements across the 
detection line): Goodson, Calonder Energy 1, and Calonder Energy 2 (Figure 4.6). STS I showed 
the lowest detection percentage (Figure 4.6). For Camrix an additional analysis that included the 
detection area (see section 5.3) showed this system detected 81.2 percent of all intrusions in the 
detection area. 
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Figure 4.6: The percentage of intrusions that were detected for each system (Camrix: before 
(89.33; additional analysis 81.82) and after (90.24; additional analysis 75.51) system 
modification). 


4.5. Discussion and Conclusions 


Downtime for most of the systems was low or non-existent. For more information about the 
causes of downtime for Calonder Energy 2, STS I and Camrix, see chapter 6. 


Different detection technologies detect large animals more or less frequently as an animal passes 
through the detection area or line of detection. This implies that care must be taken in evaluating 
the reliability of different technologies, and in comparing them to other systems or minimum 
performance requirements. 


The average number of valid detections per hour was highest for Camrix, followed by the Xtralis 
systems. The detection technologies of these systems all allow for multiple detections when an 
animal passes the line of detection or walks into the detection area. Such detections typically are 
highly clustered in time, and a relatively high number of valid detections per hour does not 
necessarily imply that such systems would result in warning signs that are activated all or most 
of the time. In addition, the number of valid detections per hour was heavily depended ion the 
test conditions; i.e. the number of animals present in the enclosure. The number of valid 
detections of wild ungulates (e.g. deer, pronghorn, elk, or moose) along a real roadside may be 
very different. 


The percentage of false positives and the average number of false positives per hour was 
relatively low for all systems (<1%; <0.10/hr). False positives do not appear to be a major 
concern with regard to the reliability of animal detection systems. 
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The percentage of false negatives (all types of false negatives combined) and the average number 
of false negatives per hour was highly variable (0-31%; 0-1.61/hr)). Note that the number of 
false negatives per hour was heavily depended ion the test conditions; i.e. the number of animals 
present in the enclosure. The number of false negatives per hour of wild ungulates (e.g. deer, 
pronghorn, elk, or moose) along a real roadside may be very different. The results suggest that 
false negatives are a major concern for some animal detection systems, but not for others. 


The percentage of intrusions (i.e., animal movements across the detection line) that were 
detected varied between 73 and 100 percent. The results suggest that false negatives are a major 
concern for some animal detection systems, but not for others. 


In conclusion, the results suggest that some animal detection systems are quite reliable in 
detecting large mammals with few false positives and false negatives, whereas the reliability of 
other systems suffers from relatively many false negatives. 
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5. INFLUENCE OF ENVIRONMENTAL CONDITIONS ON SYSTEM 
RELIABILITY 


Authors: Marcel P. Huijser', Tiffany D. Holland', Mark C. Greenwood’, Matt Blank’ & Shaowei 
Wang! 


' Western Transportation Institute, College of Engineering, Montana State University 
* Department of Mathematical Sciences, Montana State University 


5.1. Introduction 


This chapter is aimed at investigating if and how environmental conditions influence the 
reliability of animal detection systems. For details on the nine systems tested see chapter 3. For 
details on the start and end date of each test period with animals present and details on how 
system reliability was measured see chapter 4. 


5.2. Detection Data Selection 


For this chapter all three detection data types were included (see also chapter 4, Table 4.2): 


e Stratified random with animals present: Three, one-hour-long sections of video were 
randomly selected for each test day for review. 


e Non-random with animals present: These time periods were chosen based on the 
occurrence of unusual detection patterns (i.e., times with an unusually high or low 
number of detections) and certain weather events. Some of these weather events included 
the occurrence of precipitation (i.e., rain, snow or sleet), extreme temperatures (hot or 
cold), and low visibility (i.e., fog, heavy rain or snow). The length of time reviewed 
depended upon the length of the event itself; the time period analyzed varied in length. 


e Non-random without animals present: After each test with horses and llamas present, 
images were recorded for an additional 12—14 days (during these days there were no 
domesticated animals present). This is the maximum number of days that could be stored 
on the hard drive. Additional non-random time periods were selected from these days. 
These time periods were also chosen based on unusual detection patterns and certain 
weather conditions. 


The data from non-randomly selected time periods increased the range of values for different 
environmental condition parameters (see next paragraph), and increased the probability that an 
effect of environmental conditions could be detected. 
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5.3. Environmental Variables 


Environmental variables consisted of weather data and the animal species (llama or horse) 
present in the detection area or crossing the detection line. Detections caused by species other 
than llamas or horses were excluded from the analyses. 


Weather data from the Lewistown Municipal Airport weather station, about 2.4 km (1.5 mi) 
distant, was entered in the database and, based on the date and time, linked to each valid 
detection, false positive, and false negative. Weather reports were typically available in one-hour 
intervals. The data generated by the weather station included: 


Date of report 

Time of report 

Station type 

Sky conditions 
Visibility—surface statute miles 
Weather type (at time of report) 
Dry bulb temperature 

Wet bulb temperature 

Dew point temperature 

Relative humidity 

Wind speed 

Wind direction 

Wind gusts 

Station pressure 

Pressure tendency 

Net three-hour change 

Sea level pressure 

Report type 

Precipitation total (since the last regular hourly report) 
Altimeter 


In addition, the researchers recorded whether it was day or night at the time of each valid 
detection, false positive, or false negative. “Day” was defined as 30 minutes before sunrise 
through 30 minutes after sunset. “Night” was 30 minutes after sunset through 30 minutes before 
sunrise. Sunrise and sunset times were reported by the Lewistown Municipal Airport weather 
station. 


5.4. Statistical Analyses 


The effect of environmental conditions on the reliability of the nine individual animal detection 
systems was investigated through a multinomial logistic regression model with Akaike’s “An 
Information Criterion” (AIC) (Akaike, 1973) with a stepwise model selection procedure to select 
the most appropriate model. 


For this chapter the researchers distinguished two types of situations: 
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e An animal is in the detection area or crosses the detection line (see chapter 3); and 
e A system erroneously indicates an animal (denoted as a False Positive or FP). 
When an animal is in the detection area or crosses the detection line, then the system can: 
e Correctly detect the animal; or 
e Fail to detect it (False Negative or FN). 
Three different types of false negatives were distinguished (see chapter 4 for details): 


e Regular false negative (FN): the animal completely crosses the detection line and is not 
detected; 


e False negative 1 (FN1): the animal lingers in the detection zone before passing through 
the line of detection and is not detected; and 


e False negative 2 (FN2): one animal lingered in the detection zone and other animals 
passed through the line of detection without being detected. 


Thus there were five different possible response categories: 
e Correct detection 
e False positive 
e Regular false negative 
e False negative | 
e False negative 2 


However, not all responses were observed for all systems. 


Overall rates of errors were not used in this analysis for three reasons: 
e The rate that each system “fires” or “detects” is different for each system (see chapter 4); 


e The detection areas differ in size and location for the different systems (see chapter 3), 
and; 


e Non-randomly selected times were included in the analyses for this chapter to increase 
the range of values for parameters describing the environmental conditions. 


Instead, system errors were related to correct detections through logistic regression models. 
Logistic regression models use categorical responses to model probabilities of success using the 
logistic link function (log(/(1-7))), which leads to modeling on the log-odds scale. If a random 
sample of a certain number of events was considered and the different systems produced the 
same number of events per time, then the probability of each type of event could be considered. 
However, the different sampling rates of the systems (see chapter 4) could lead a system with a 
higher rate to record a single animal crossing 10 times whereas another system might only record 
one animal crossing. Even if the success rate is the same in both systems, the opportunities for 
correct or incorrect detections would be different in the different systems. Also, since the correct 
detection rate depends on the animal’s path through the enclosure, comparing probabilities 
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between systems is not reasonable. However, an advantage of logistic models is that log-odds 
scale slope coefficient is unaffected by varying sampling rates either in the explanatory or 
response categories. 


Consider the following exaggerated and unrealistic example in Table 5.1 where the odds of 
correct detection between day and night are the same in both systems, in that they are twice as 
high during the day than during the night. But for some reason, ten times more correct 
identifications were observed in one system, either due to animal path choices or system “firing” 
or detection rates. Using a logistic regression model, log(1/(1-7))=Bo + BiDay, both tables result 
in the same estimate of 8, but different estimates of Bo and different estimates of m based on the 
different intercept values. B; provides the log-odds of success during the day compared to the 
night and exp(P;) provide the odds ratio comparing day and night. The estimate of the intercept 
term, Bo, depends on the number of correct identifications observed. This simple example 
motivates the focus throughout the modeling process only on slope coefficients, as this same 
argument applies to all the multinomial models considered. Another feature of this example is 
that the standard error associated with the second data set is smaller than for the first example, so 
there is a benefit to including additional information even though it would not change the point 
estimates for the coefficients. 


Table 5.1: The number of hours that detection data were available for analyses for each system. 


System 1 System 2 
False Correct False Correct 
positives | detections | positives | detections 
Day 5 10 5 100 
Night 5 5 5 50 


Multinomial logistic regression models were used due to the multi-category nature of the 
response variable that had up to five possible categories. One version of these models is called 
the baseline category model (Agresti, 2007) where one category is chosen as a baseline or 
reference category and then up to four typical logistic regression models are estimated to predict 
the difference between the category of interest and the baseline category. Positive (or negative) 
coefficients provide higher (or lower) log-odds of being the category of interest relative to the 
baseline category. Here the baseline category was chosen to be a correct identification and each 
sub-category logit model is focused on predicting each type of error relative to a correct 
identification. 
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A simple example using only Day/Night as an explanatory variable leads to the following 
multinomial logit model: 


TW 

loo 2 = Qo) + By, Day 
TW 

oo =a, + B,,Day 


joo = Ay) + B,,Day 


1 
oof = A, + B;,Day. 


In the previous model, mo corresponds to FN, 7 to FN1, m2 to FN2, 73 to FP, and 74 to a correct 
detection. Each row in the model is a “sub-category” logistic regression model. The only 
coefficients of interest in interpreting this model would be for the effect of day in the transition 
or comparison between log-odds of errors in the night versus the day. On the log-odds scale, 
positive-valued effects correspond to higher rates of an error for day than night and negative 
coefficients flip the effect around. A coefficient close to 0 would suggest that there is negligible 
day/night effect. We can judge closeness to 0 using a test statistic instead of the magnitude of the 
coefficients since it adjusts for the variability in the estimate. If the test statistic is small, then 
there is little evidence that that coefficient should be different from 0. A cut-off of +2 was used 
below to focus the interpretation on coefficients that look to be different from 0. Note, however, 
that it is possible to have an overall effect that is significant in an ANOVA (here it would be an 
analysis of deviance) type test where all of the coefficients involved in that effect would not meet 
this cut-off. 


Based on the assumed multinomial distribution for the response variable, a multinomial 
distribution is used to define a likelihood. This likelihood is maximized to provide parameter 
estimates and associated standard errors of those estimated coefficients. These are interpreted 
without reference to specific probabilities of events to allow equal comparison across systems 
and allow for some non-randomly selected times to be used to augment the randomly sampled 
information. 


An additional advantage of this modeling perspective is that it is possible to consider different 
models for each sub-category model. This is particularly important when false positives are 
considered along with the explanatory variable of type of animal. It is only possible to get false 
positives where there are no animals present to be detected and this uninformative model must 
not be considered. This implementation of multinomial logit models is available via the VGAM 
package (Yee, 2008) in R with the interface to these methods performed using the Zelig package 
(Imai, 2008). 


The following variables were considered in step-down AIC-driven model selection to generate a 
set of candidate models to compare AIC values. Models within two AIC units of the top model 
were considered for selection. Within these constraints, the selection process focused on the 
simplest model that contained the variables that were present in most of the models within two 
AIC units of the top model. The units or categories for each variable are given between brackets. 
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For categorical variables the effect is calculated in relation to a “standard” category). For 
example, for wind direction, the effect of northern, southern or western winds is calculated 
compared to eastern winds). Similarly, the effect of high winds is calculated by comparing the 
presence of high winds to the absence of high winds. 


e Wind direction (split into 4 categories for N, S, W/E) 

e Wind speed (mi/h) 

High wind (winds over 15 mph) (present/absent) 

Wind gust (present/absent) 

Temperature (°C) 

Day or night (Day: 30 min before sunrise until 30 min after sunset; Night: 30 min after 
sunset until 30 min before sunrise) 

e Visibility (excellent: >10 mi, less-than excellent: <10 mi) 
e Relative humidity (%) 

e Precipitation (present/absent) 

e Animal (none, horse or llama) 


The three variables related to wind velocity (wind speed, high wind, and wind gust) were 
considered individually in each model. Considering wind speed and high wind and wind gust 
together is unreasonable as they are highly correlated and can be considered as different 
transformations of similar information. Animal is problematic for typical multinomial logistic 
models as noted above as the “none” category is associated with false positives by definition. But 
the difference in “animal” is important to consider for the other types of events. To incorporate 
this effect only where it is reasonable, it is only used for the subcategory logit models for false 
negatives (FN, FN1, FN2), and not for false positives (FP). 


In some situations, wind direction was not defined due to low wind speeds. In these situations, a 
randomly selected direction from the observed directions was generated to impute each missing 
observation. This retains approximately the same distribution of wind directions that were 
observed but prevents the models from encountering missing information. Multiple runs through 
the imputation were considered for some of the systems and the differences in the model 
selection and coefficient estimates were negligible across the runs, with coefficients changing in 
the second decimal point generally characterizing those results. By randomly imputing those 
missing values, wind direction should have less of a chance of being a useful explanatory 
variable, but it was included in the model for many of the systems even with the imputation. 


Since each variable is retained across all sub-category models (except for “‘animal”), the effect 
must either be large in one model or somewhat useful across the different models to be selected 
by AIC. Further simplification would be possible if this condition would be relaxed, but the 
complexity of the model selection process would be exponentially higher for modeling each 
system if model selection was considered for each sub-category logit model. Since typical 
multinomial models do not allow this degree of flexibility in modeling, this choice retains 
comparability to more conventional multinomial logit modeling with the only difference from 
these typical models involved in the false positive sub-category logit and the animal explanatory 
variable. 


Models for systems that had their settings or sensitivity modified at some point between the test 
periods had a variable included to address such changes, as the purpose of such modifications 
was to reduce the occurrence of errors. The inclusion of this “modification” variable 
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(before/after) in the models was mandatory (forced) (Table 5.2). Systems that did not have their 
settings or sensitivity modified between the test periods did not have the “modification” variable 
included in their models. 


Some systems had very low numbers for certain types of errors or did not display certain types of 
errors at all. If the frequency of a certain type of error was <3, the type of error concerned was 
excluded from the models. 


To simplify the interpretation of the vast number of parameters in the models, only coefficients 
that have z test statistics over two (P<0.05) are considered for interpretation. This is not a testing- 
based approach to interpret the coefficients, as a variable could be significantly included in the 
model and not have any significant coefficients. It is simply used to highlight the most important 
features of the models. 


Table 5.2: Dates modifications were made to each system (see also chapter 6). 


System Manufacturer ID Dates modifications 
number and system name number 
(Figure 
3.2) 
1 Xtralis 7 29 November 2007 
(ADPRO) 
2 Xtralis 5-6 29 November 2007 
(ADPRO) 
3 STS 1 No modifications 
(RADS I) 
4 STS 2 No modifications 
(RADS II) 
5 Calonder Energy (CAL 1 No modifications 
92, LS-WS-WE 45) 
6 Calonder Energy (CAL 2 No modifications 
92, IR-204-319/M3) 
7 Camrix 29 November 2007 
(A.L.E.R.T.) 
8 Xtralis 1-2 29 November 2007 
(ADPRO) 
9 Goodson No modifications 
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With this experiment running over time, there is some concern about clustered or correlated 
responses. Highly correlated responses can cause overdispersion, which is where the variability 
in the generalized linear model exceeds the amount that was assumed based on the model. In 
those situations, the likelihood and standard errors are not accurate. It is possible to incorporate 
an adjustment to the likelihood based on an estimate of overdispersion leading the QAIC 
(Burnham & Anderson, 2003) and to inflate standard errors in a similar way. Adjustments for 
overdispersion are suggested when Pearson’s X” or the residual deviance test for lack of fit for 
the “fullish” model are much larger than their respective degrees of freedom. The “fullish” 
model is based on the most complicated model considered in the candidate models before any 
model reduction is considered. The degrees of freedom for the k category multicategory logit 
models are (n*(k-1)-total # parameters in the model). The deviance was compared to its df for 
each system. 


5.5. Results 


The number of errors and correct detections included in the analyses are shown in Table 5.3. As 
discussed in the methods section, some systems had very low numbers for certain types of errors 
or did not display certain types of errors at all. If the frequency of a certain type of error was <3, 
the type of error concerned was excluded from the models. Note that this left insufficient data to 
conduct the analysis for the Calonder Energy 2 system. Also note that the errors listed in Table 
5.3 are based on a combination of randomly and non-randomly selected time periods and that the 
error counts in the table cannot be used to compare systems. 
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Environmental conditions 


Table 5.3: The number of errors and correct detections for each system. 


System Manufacturer ID False False False False Correct 
number and system name number | negatives | negatives | negatives | positives | detections 
Giawe (FN) 1 (FN1) 2 (FN2) (FP) 
3.2) 
1 Xtralis 7 51 36 30 0 1,020 
(ADPRO) 
2 Xtralis 5-6 81 104 47 2 879 
(ADPRO) 
3 STS 1 100 71 16 0 418 
(RADS I) 
4 STS 2 21 12 a 0 211 
(RADS II) 
5 Calonder Energy (CAL | 1 3 0 0 30 622 
92, LS-WS-WE 45) 
6 Calonder Energy (CAL | 2 1 0 0 0 100 
92, IR-204-319/M3) 
7 Camrix 38 38 114 190 439 
(A.L.E.R.T.) 
8 Xtralis 1-2 21 41 8 42 1,002 
(ADPRO) 
9 Goodson 1 0 0 666 643 


The model selected for each system is indicated in Table 5.4. Shaded cells indicate the parameter 
concerned was included in the selected model whereas unshaded cells indicate the parameter 
concerned was excluded in the selected model. None of the selected models for the systems 


required an adjustment for overdispersion. 
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Table 5.4: The effect of environmental conditions on system reliability (see text). 


2 2 B 3 2 
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3 E a: |e) 3 £ = - 2 ‘ g = 
|| eee tails = 2g | s | | 2/8] ¥ 
5 z & | § & 5 e 2 3 2 g 
a A = mils = = a > x & < 
Xtralis 7 +FN +FN 
+FN1 +FN1 
+FN2 +FN2 -FN2 
FP 
Xtralis 5-6 +FN +FN -FN +FN +FN 
+FNI1 (N/E) -FN1 +FN1 
+FN2 -FN2 (N/E) +FN2 +FN2 
FP 
STS 1 +FN -FN +FN 
-FN1 
FP 
STS 2 +FN 
+ENI (S/E) +FN1 
+FN2 +FN2 +FN2 
FP 
Calonder Energy 1 FN 
FN1 
FN2 
nel +FP -FP eRe, 
Camrix +FN +FN 
-FN1 +FN1 
-FN2 +FN2 +FN2 
-FP +EP (N/E) -FP +FP 
Xtralis 1-2 +FN +FN +FN 
-FN1 (N/E) -FN1 +FN1 
+FN2 
-FP 
Goodson FN 
FN1 
FN2 
-FP | +FP(N+W+S/E) | -FP -FP 
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Table 5.4 has four rows for each system. Each row relates to a specific type of error. The first 
row relates to regular false negatives (FN), the second to false negatives 1 (FN1), the third to 
false negatives 2 (FN2), and the fourth to false positives (FP). If a shaded cell contains a symbol 
for one of these error types (FN, FN1, FN2, FP), the environmental parameter concerned had a 
significant effect (P<0.05) on that type of error. The direction of the effect is indicated through — 
and + symbols. The results for each system are discussed in detail below. 


5.5.1. | System 1 (Xtralis (ADPRO) 7) 


The selected model excluded false positives. System modification did not have a significant 
effect on the reliability of the system. An increase in wind speed was associated with an increase 
in FN, FN1, FN2 (log-odds of a FN, FN1 and FN2 relative to a correct identification increase by 
0.08, 0.13, and 0.07 respectively for each mi/h increase). An increase in temperature was 
associated with an increase in FN and FN2 (log-odds of a FN and FN2 relative to a correct 
identification increase by 0.05 and 0.06 respectively for each 1°C increase). An increase in 
relative humidity was associated with a decrease in FN2 (log-odds of a FN2 relative to a correct 
identification decrease by -0.05 for each 1% increase in relative humidity). Finally, llamas 
(compared to horses) were associated with an increase in FN1 (log-odds of a FN1 relative to a 
correct identification increase by 1.86 for a llama compared to a horse). 


5.5.2. System 2 (Xtralis (ADPRO) 5-6) 


The selected model excluded false positives. System modification did not have a significant 
effect on the reliability of the system. An increase in wind speed was associated with an increase 
in FN and FN2 (log-odds of a FN and FN2 relative to a correct identification increase by 0.10 
and 0.11 respectively for each mi/h increase). Wind coming from the north (compared to wind 
from the east) was associated with more FN1 and fewer FN2 (log-odds of a FN1 relative to a 
correct identification increase by 0.56 with northern winds compared to eastern winds; log-odds 
of a FN2 relative to a correct identification decrease by -1.49 with northern winds compared to 
eastern winds;). An increase in temperature was associated with an increase in FN and FN2 (log- 
odds of a FN and FN2 relative to a correct identification increase by 0.10 and 0.11 respectively 
for each 1°C increase). Nights (compared to days) were associated with fewer FN and FN1 (log- 
odds of a FN and FN1 relative to a correct identification decreased by -0.84 and -1.02 during the 
night). An increase in relative humidity was associated with an increase in FN (log-odds of a FN 
relative to a correct identification increase by 0.06 for each 1% increase in relative humidity). 
Finally, llamas (compared to horses) were associated with an increase in FN, FN1, and FN2 (log- 
odds of a FN, FN1, and FN2 relative to a correct identification increase 1.70, 1.33, and 1.11 
respectively for a llama compared to a horse). 


5.5.3. System 3 (STS (RADS) 1) 


The selected model excluded false positives. Higher temperatures and higher relative humidity 
were associated with an increase in FN (log-odds of a FN relative to a correct identification 
increase by 0.03 for each 1°C increase; log-odds of a FN relative to a correct identification 
increase by 0.02 for each 1% increase in relative humidity). Nights (compared to days) were 
associated with fewer FN and FN1 (log-odds of a FN and FN] relative to a correct identification 
decrease by -1.57 and -1.86 during the night). The species (llama or horse) did not have a 
significant effect on the reliability for this system. 
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5.5.4. System 4 (STS (RADS) 2) 


The selected model excluded false positives. Wind coming from the south (compared to wind 
from the east) was associated with more FN1 (log-odds of a FNI relative to a correct 
identification increase by 2.69 with southern winds compared to eastern winds). Higher 
temperatures were associated with an increase in FN1 and FN2 and higher relative humidity was 
associated with an increase in FN2 (log-odds of a FN1 and FN2 relative to a correct 
identification increase by 0.28 and 0.30 respectively for each 1°C increase; log-odds of a FN2 
relative to a correct identification increase by 0.28 for each 1% increase in relative humidity). 
Finally, lamas (compared to horses) were associated with an increase in FN and FN2 (log-odds 
of a FN and FN2 relative to a correct identification increase by 1.67 and 2.77 respectively for a 
llama compared to a horse). 


5.5.5. | System 5 (Calonder Energy 1 (CAL 92, LS-WS-WE 45)) 


The selected model excluded all types of false negatives. Since the model only related to false 
positives, the animal species variable was excluded from the model. The presence of wind gusts, 
an increase in temperature, and an increase in relative humidity were all associated with an 
increase in false positives (log-odds of a FP relative to a correct identification increase by 1.92 
with wind gusts compared to no wind gusts; log-odds of a FP relative to a correct identification 
increase by 0.11 for each 1°C increase; log-odds of a FP relative to a correct identification 
increase by 0.06 for each 1% increase in relative humidity). Excellent visibility (compared to 
less-than-excellent visibility) was associated with a decrease in false positives (log-odds of a FP 
relative to a correct identification decrease by -4.29 with excellent visibility compared to less- 
than-excellent visibility). 


5.5.6. | System 6 (Calonder Energy 2 (CAL 92, IR-204-319/M3)) 


There were insufficient data to conduct an analysis for this system (see table 5.3). 


5.5.7. System 7 (Camrix (A.L.E.R.T.) 


System modifications were associated with a decrease in FN2 (log-odds of a FN2 relative to a 
correct identification decrease by -1.52 after system modification). An increase in wind speed 
was associated with an increase in FN, and a decrease in FP (log-odds of a FN relative to a 
correct identification increase by 0.07 for each mi/h increase; log-odds of a FP relative to a 
correct identification decrease by -0.19 for each mi/h increase). Wind coming from the north 
(compared to wind from the east) was associated with more FP (log-odds of a FP relative to a 
correct identification increase by 1.51 with northern winds compared to eastern winds). An 
increase in temperature was associated with a decrease in FN1 and FP (log-odds of a FN1 and FP 
relative to a correct identification decrease by -0.06 and -0.08 respectively for each 1°C 
increase). Nights (compared to days) were associated with an increase in FN, FN1, and FN2 
(log-odds of a FN, FN1, and FN2 relative to a correct identification increase by 2.27, 1.05, and 
1.94 respectively during the night). Excellent visibility (compared to less-than-excellent 
visibility) was associated with an increase in FN2 (log-odds of a FN2 relative to a correct 
identification increase by 2.46 with excellent visibility compared to less-than-excellent 
visibility). An increase in relative humidity was associated with an increase in FP (log-odds of a 
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FP relative to a correct identification increase by 0.07 for each 1% increase in relative humidity). 
The species (llama or horse) did not have a significant effect on reliability for this system. 


5.5.8. System 8 (Xtralis (ADPRO) 1-2) 


System modification did not have a significant effect on the reliability of the system. Wind 
coming from the north (compared to wind from the east) was associated with a decrease in FN1 
(log-odds of a FN1 relative to a correct identification decrease by -1.09 with northern winds 
compared to eastern winds). An increase in temperature was associated with an increase in false 
negatives (log-odds of a FN relative to a correct identification increase by 0.14 for each 1°C 
increase). Nights (compared to days) were associated with a decrease in FN1 (log-odds of a FN1 
relative to a correct identification decrease by -1.75 during the night). An increase in relative 
humidity was associated with an increase in FN and a decrease in FP (log-odds of a FN relative 
to a correct identification increase by 0.06 for each 1% increase in relative humidity; log-odds of 
a FP relative to a correct identification decrease by -0.05 for each 1% increase in relative 
humidity). Finally, llamas (compared to horses) were associated with an increase in FN, FN1, 
and FN2 (log-odds of a FN, FN1, and FN2 relative to a correct identification increase 1.35, 1.29, 
and 2.88 respectively for a llama compared to a horse). 


5.5.9. | System 9 (Goodson) 


The selected model excluded all types of false negatives. Since the model only related to false 
positives, the animal species variable was excluded from the model. The presence of wind gusts, 
an increase in temperature, and an increase in relative humidity were all associated with a 
decrease in FP (log-odds of a FP relative to a correct identification decrease by -1.36 with wind 
gusts compared to no wind gusts; log-odds of a FP relative to a correct identification decrease by 
-0.18 for each 1°C increase; log-odds of a FP relative to a correct identification decrease by -0.02 
for each 1% increase in relative humidity). However, the effect of wind gusts is likely an artifact 
of the sampling design as the majority of the false positives happened to occur when wind gusts 
were not present. Wind coming from the north, south, and west (compared to wind from the east) 
was associated with an increase in FP (log-odds of a FP relative to a correct identification 
increase by 1.23, 1.41, and 1.11 respectively with northern, southern, and western winds 
compared to eastern winds). 


5.6. Discussion and Conclusions 


Wind gusts or wind speed were included in all models, suggesting wind is an important factor in 
the reliability performance of the systems. High winds were associated with an increase in 
different types of false negatives for most passive infrared area-cover systems (except Xtralis 1- 
2). High winds were associated with both an increase in false positives (Calonder Energy 1) and 
a decrease in false positives (Camrix). This suggests that passive infrared area-cover systems 
become less sensitive with high winds whereas break-the-beam systems that rely on a very 
narrow beam (in this case a laser beam) may start generating false positives, presumably because 
the sensors sway slightly in and out of alignment. The latter suggests the importance of a stable 
foundation and pole for break-the-beam systems. Stable foundations and poles may also be 
beneficial to passive infrared area-cover systems, but it is unclear if the increase in false 
negatives for such systems is caused by movement of the sensors that tend to be higher up on a 
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pole than sensors for break-the-beam systems, or by vegetation or pockets of hot and cold air that 
move in the wind. 


Wind direction was present in six of the eight models, suggesting wind direction is an important 
factor in the reliability performance of the systems. The effects are hard to interpret, but it may 
be that winds oriented perpendicular to the systems caused vegetation or pockets of hot and cold 
air to trigger systems more often than winds oriented more parallel to the systems. 


The temperature variable was present in all selected models suggesting temperature is an 
important factor in the reliability performance of the systems. Higher temperatures are generally 
associated with higher error rates. This could be due to temperature causing reduced performance 
of the equipment. In addition, passive infrared systems may not be able to distinguish clearly 
between pockets of hot air and moving animals. However, higher temperatures are concentrated 
in time (summer) and it is possible that factors other than temperature caused more errors in 
summer. Animal behavior and possible effects on the likelihood of correct detections and errors 
may have also been influenced by temperature. 


The day and night variable was present in six of the eight selected models suggesting the effect 
of day and night is an important factor in the reliability performance of the systems. Three 
systems (Xtralis 5-6, STS 1, and Xtralis 1-2) had fewer false negatives during the night 
compared to during the day. This may be related to lower temperatures or higher contrasts in 
temperatures of the animals and their surroundings during the night. However, Camrix had more 
false negatives during the night compared to during the day. 


The visibility variable was included in four of the eight selected models suggesting visibility is 
an important factor in the reliability performance of some systems. Excellent visibility was 
associated with fewer false positives for a break-the-beam system (Calonder Energy 1), which 
suggests that relatively low visibility may block or reduce the narrow signal path of optical 
break-the-beam systems. It is unclear why excellent visibility may have caused the Camrix 
system to increase false negatives 2. 


Precipitation was rarely observed during the test periods. Thus the fact that this variable was only 
present in two of the eight selected models is probably more related to the local conditions 
(generally dry climate) than the actual effect of precipitation. This can be contrasted by the 
importance of relative humidity as that parameter was included in all selected models suggesting 
humidity (and thus probably also precipitation) is an important factor in the reliability 
performance of the systems. Higher relative humidity was generally associated with an increase 
in errors. However, some systems showed a decrease in errors with increased relative humidity. 


The animal species parameter (llamas vs. horses) was included in all models that included false 
negatives suggesting that the size of the target species is an important factor in the reliability 
performance of the systems. Llamas were substantially harder to detect for most systems, 
especially passive infrared area-cover systems, than horses, probably because of their smaller 
body size. 
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6. EXPERIENCES WITH INSTALLATION, OPERATION, AND 
MAINTENANCE 


Authors: Tiffany D. Holland, Marcel P. Huijser, Matt Blank & Shaowei Wang, Western 
Transportation Institute, College of Engineering, Montana State University 


6.1. System 1 (Xtralis (ADPRO) 7) 


Xtralis 7 was installed on September 21, 2006. 


On July 3, 2007, two error messages were observed when the computer in Lewistown was 
accessed remotely from the WTI/MSU office in Bozeman. The error messages read: “Integer- 
Uberlauf’ and “Exception EIntOverflow im Modul ASIM-S.exe bei 000498BC. Integer- 
Uberlauf.” The system was restarted, and when researchers were in Lewistown on July 18, 2007, 
the system was operating properly. However, there was no data available from June 30, through 
July 18, 2007. 


The system stopped recording detections on September 29, 2007. The system was restarted on 
November 16, 2007 and began detecting properly. 


At the request of Xtralis, the system was upgraded and made more sensitive on November 29, 
2007. 


Xtralis sensor 1 was not detecting properly on January 18, 2008. While trouble-shooting, an error 
message appeared, and the computer was restarted. After restarting the computer, Xtralis 7 was 
not found or recognized by the computer, and another error message appeared. It read 
“Information: Sorry no Detector found!” On January 22, 2008, the system was accessed via the 
remote connection in Bozeman. Xtralis 7 was previously assigned to COM port 2. However, the 
system was automatically reassigned to COM port 10. Settings were adjusted to accommodate 
the new COM port, and the computer was then able to locate the sensor. 


After power outages unrelated to the system itself, the data logger shut down and there were no 
data available for the following dates: February 21—27, 2007, March 23—26, 2007, April 21—23, 
2007, and April 26— May 4, 2007. There were also no data available February 19-20, 2007, 
March 2-9, 2007, November 21—29, 2007, and December 12, 2007— January 18, 2008. This was 
likely due to power outages. 


6.2. System 2 (Xtralis (ADPRO) 5-6) 


Xtralis 5—6 was installed on September 21, 2006. 


On April 19, 2007, during a routine check, researchers noticed the Xtralis 6 sensor was not 
detecting properly. Snow was cleared off the lens of the sensor, and it began operating properly 
again. 


On July 3, 2007, two error messages were observed when the computer in Lewistown was 
accessed remotely from Bozeman. The error messages read: “Integer-Uberlauf” and “Exception 
EIntOverflow im Modul ASIM-S.exe bei 000498BC. Integer-Uberlauf.” The system was 
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restarted, and when researchers were in Lewistown on 18 July 2007, the system was operating 
properly. However, there was no data available from June 30 through July 18, 2007. 


The system stopped recording detections on September 29, 2007. The system was restarted on 
November 16, 2007 and began operating properly again. 


At the request of Xtralis, the system was upgraded and made more sensitive on November 29, 
2007. 


Xtralis sensor 1 was not detecting properly on January 18, 2008. While trouble-shooting, an 
error message appeared, and the computer was restarted. After restarting the computer, neither 
sensor (Xtralis 5 or Xtralis 6) was found or recognized by the computer, and another error 
message appeared. It read “Information: Sorry no Detector found!” On January 22, 2008, the 
system was accessed via the remote connection in Bozeman. Xtralis 5—6 was previously assigned 
to COM port 2. However, the system was automatically reassigned to COM port 10. Settings 
were adjusted to accommodate the new COM port, and the computer was then able to locate both 
sensors. 


After power outages unrelated to the system itself, the data logger shut down and there were no 
data available for the following dates: February 21—27, 2007, March 23-26, 2007, April 21—23, 
2007, and April 26—May 4, 2007. There were also no data available February 19-20, 2007, 
March 2-9, 2007, November 21—29, 2007, and December 12, 2007—January 18, 2008. This was 
likely due to power outages. 


6.3. System 3 (STS (RADS) 1) 


STS 1 was installed on October 19, 2006. 


On April 19, 2007, researchers noticed during a routine check that STS 1 was not detecting 
properly, though it was still communicating with the computer and data storage software. 


Also on April 19, 2007, a new antenna was installed outside of the office to improve 
communication between the system and the computer. 


On August 31, 2007, while checking all systems via remote access from Bozeman, it was noted 
that the system was not recording data properly. The data logger software was restarted but still 
did not record data. When in Lewistown on September 4, 2007, an indicator light signifying that 
the entire system shut down due to a power interruption was observed. The memory card was 
changed, the system was restarted, and the system began operating and storing data properly 
again. As a result of this power interruption, there were no data available from August 22, 2007 
through September 4, 2007. 


On September 4, 2007, vegetation was removed from in front of the system to prevent false 
positives due to tall and moving vegetation. 


The system stopped recording detections on November 4, 2007. The system was restarted on 
November 16, 2007, however, the system still did not detect properly. 


On November 29, 2007, researchers attempted to repair the system by adjusting the gain and 
checking connections. On December 4, 2007, the Digital Signal Processor (DSP) card was 
replaced and voltages and the gain settings were checked. The Multi Media Card (MMC) card 
was also swapped, and the system was restarted. The “ribbon connections” were checked and 
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the Microcontroller (MTC) board was cleaned on January 18, 2007, and the system began 
operating properly. 


After power outages unrelated to the system itself, the data logger shut down and there were no 
data available for the following dates: February 19-27, 2007, March 24—25, 2007, April 22—23, 
2007, and April 26—May 4, 2007. There were also no data available November 4-16, 2007, 
November 20-29, 2007, and December 12, 2007— January 18, 2008. This was likely due to 
power outages. 


6.4. System 4 (STS (RADS) 2) 


STS 2 was installed on July 18-19, 2007. 


On August 31, 2007, while checking all systems via remote access from Bozeman, it was noted 
that the system was not recording data properly. The data logger software was restarted but still 
did not record data. When in Lewistown on September 4, 2007, an indicator light signifying that 
the entire system shut down due to a power interruption was observed. The memory card was 
changed, the system was restarted, and the system began detecting and storing data properly 
again. As a result of this power interruption, there were no data available from August 22, 2007 
through September 4, 2007. 


On September 4, 2007, vegetation was removed from in front of the system to prevent false 
positives due to tall and moving vegetation. 


The system stopped recording detections on November 4, 2007. The system was restarted on 
November 16, 2007, and the system began operating properly. 


There were no data available November 4—16, 2007, November 20—29, 2007, and December 12, 
2007—January 18, 2008. This was likely due to power outages. 


No data were available for part of the day on December 4, 2007, due to work being done on the 
system to repair STS 1. 


6.5. System 5 (Calonder Energy 1 (CAL 92, LS-WS-WE 45)) 


Calonder Energy 1, a laser break-the-beam system, was installed on September 21—22, 2006. 


On July 18, 2007, researchers observed that the system was properly detecting, as indicated by 
the counter attached to the actual sensor. However, there was a communication problem between 
the system and the computer and data software, and researchers were unable to download 
detection data. 


On August 22, 2007, it was determined that the communication problem between the system and 
the computer was caused by the data logger. The data logger was removed and was sent to 
Calonder for repair. 


The data logger was reinstalled on November 16, 2007. The system began recording data and 
continued to detect properly. Due to this error, there are no detection data available from June 6 
through November 16, 2007. 
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There are also no data available for April 22-25, 2007 and May 24-26, 2007. Researchers 
believe this may also be related to the data logger malfunction described above. 


On January 18, 2008, the system was not detecting properly. Ice was melted off of the receiver, 
and the system then operated properly again. 


6.6. System 6 (Calonder Energy 2 (CAL 92, IR-204-319/M3)) 


Calonder Energy 2, a passive infrared system, was installed on September 21—22, 2006. 


On March 16, 2007, researchers noticed that CE 2 was not properly recording detections. It was 
left in place during the next test (March 16—25, 2007) to confirm that it was not properly 
detecting animals. The sensor was removed on April 19, 2007 and sent back to Switzerland for 
repairs, having not worked since January 2007. The cover on the original system was installed 
incorrectly, and water caused damage to the internal electronics. A replacement sensor was 
received by WTI/MSU in June 2007. 


On July 18-19, 2007, the system was reinstalled; however, it was still not performing properly. 
The system was detecting in a 2.5—3.0 m (8.2—9.8 ft) window approximately 12 m (39 ft) from 
the sensor, as opposed to the expected 88—90 m (289-295 ft) detection area beginning 10-12 m 
(33-39 ft) from the sensor. 


Also on July 18-19, 2007, researchers observed that, in addition to the system not properly 
detecting, there was also a communication problem between the system and the computer and 
data software. Researchers were unable to download detection data. 


On July 30, 2007, without further attempt to install the system properly, the detection area had 
increased to approximately 60 m (197 ft), beginning about 12 m (39 ft) from the system. 
However, this was still not the expected detection area of 88—90 m (289-295 ft). 


After receiving further installation instructions from Calonder, the system was properly 
reinstalled on August 22, 2007. 


On August 22, 2007, it was determined that the communication problem between the system and 
the computer was caused by the data logger. The data logger was removed and was sent to 
Calonder for repair. 


The data logger was reinstalled on November 16, 2007. The system began recording data and 
continued to detect properly. As a result of the data logger malfunction, there are no detection 
data available from August 22, 2007 through November 16, 2007. 


The counter in the box was reset on January 18, 2008, as it was at its maximum capacity of 
12,000 counts. However, the counter being at its maximum had no effect on the performance of 
the system. 


6.7. System 7 (Camrix A.L.E.R.T.) 


Camrix installation began on October 19, 2006 and was completed on October 31, 2006. 


On March 21, 2007, during a test (March 16—25, 2007), the Camrix computer “froze” with an 
image of an animal. When the test was ended on March 26, 2007, the system was not detecting 
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properly, and the “frozen” image was observed. The computer, and therefore the Camrix 
software, was restarted and the system began operating properly again. 


On June 5, 2007, though the system was detecting properly, a warning message was present. It 
stated “Microsoft Windows warning: the system has recovered from a serious error.” The 
computer, and therefore the Camrix software, was restarted, and the system continued operating 


properly. 
On July 28, 2007, the Camrix system “froze” again during a test (July 20-30, 2007). On July 30, 


2007, the system was not detecting properly, and a “frozen” image was observed. The computer, 
and therefore the Camrix software, was restarted, and the system began operating properly again. 


On November 16, 2007, though the system was detecting properly, a warning message was 
present. It stated “Microsoft Windows warning: the system has recovered from a serious error.” 
The computer, and therefore the Camrix software, was restarted, and the system continued 
operating properly. 


At the request of Camrix, an upgrade to the system was installed on November 29, 2007. 


6.8. System 8 (Xtralis (ADPRO) 1-2) 


Xtralis 1-2 was installed on August 8, 2006. 


On July 3, 2007, two error messages were observed when the computer in Lewistown was 
accessed remotely from Bozeman. The error messages read: “Integer-Uberlauf” and “Exception 
EIntOverflow im Modul ASIM-S.exe bei 000498BC. Integer-Uberlauf.” The system was 
restarted, and when researchers were in Lewistown on July 18, 2007, the system was operating 
properly. However, there was no data available from June 30 through July 18, 2007. 


The system stopped recording detections on September29, 2007. The system was restarted on 
November 16, 2007 and then began operating properly again. 


At the request of Xtralis, the system was upgraded and made more sensitive (lower thresholds for 
detections) on November 29, 2007. 


Xtralis sensor 1 was not detecting properly on January 18, 2008. While trouble-shooting, an error 
message appeared and the computer was restarted. After restarting the computer, neither sensor 
(Xtralis 1 or Xtralis 2) was found or recognized by the computer, and another error message 
appeared. It read “Information: Sorry no Detector found!” On January 22, 2008, the system was 
accessed via the remote connection in Bozeman. Xtralis 1-2 was previously assigned to COM 
port 2. However, the system was automatically reassigned to COM port 10. Settings were 
adjusted to accommodate the new COM port, and the computer was then able to locate both 
sensors. 


After power outages unrelated to the system itself, the data logger shut down and there were no 
data available for the following dates: February 21—27, 2007, March 23-26, 2007, April 21—23, 
2007, and April 26— May 4, 2007. There were also no data available February 19-20, 2007, 
March 2-9, 2007, November 21—29, 2007, and December 12, 2007—January 18, 2008. This was 
likely due to power outages. 
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6.9. System 9 (Goodson) 


Goodson was installed in December 2006. 


On March 2, 2007, researchers noticed that the loose ends of the straps used to secure the system 
to the pole may have been blowing in front of the detection beam. This may have been the cause 
of false positives previously observed. These straps were secured so that no false positives could 
occur in this manner. As a result, the system was not analyzed for false positives prior to March 
2, 2007. 


On April 19, 2007, researchers noticed the Goodson system was detecting properly but was also 
recording many false positives. This may have been related to environmental conditions such as 
the angle of the sun, or other conditions; the temperature was about 2°C (35°F) with a varying 
wind of about 16 to 48 km (10 to 30 mi) per hour, and snowfall was reported. 


On July 18, 2007, researchers observed the low battery indication, which was initially reported 
on July 3, 2007. New batteries were installed in both the transmitter and receiver. 


On September 4, 2007, vegetation was removed from in front of the system to prevent false 
positives due to tall and moving vegetation. 


On January 18, 2008, a low-battery indication was observed, which was initially reported on 
January 15, 2008. New batteries were installed in both the transmitter and receiver. In addition, 
ice was melted off of the transmitter. 


From November 22 through November 29, 2007, the data logger on the receiver was at its 
maximum capacity. Therefore, no data were recorded during that time. 
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7. NATIONAL ITS ARCHITECTURE AND STANDARDS 


Authors: Pat T. McGowen, Barrett Hubbard & Marcel P. Huijser, Western Transportation 
Institute, College of Engineering, Montana State University 


7.1. What is ITS Architecture? 


The Transportation Equity Act for the 21st Century (TEA-21), requires that federally funded ITS 
projects conform to the National ITS Architecture standards. As defined in TEA-21, the term 
“intelligent transportation system” means “electronics, communications, or information 
processing used singly or in combination to improve the efficiency or safety of a surface 
transportation system” (AASHTO et al., 2002). 


National ITS Architecture standards govern both the functions performed in implementing ITS, 
and the information flows between transportation subsystems. Key requirements of these 
regulations are that regional ITS architectures must be prepared, all ITS projects must follow a 
systems engineering process, and that ITS standards be used (AASHTO et al., 2002). 


The ITS National Architecture’s purpose is to make sure that all intelligent transportation 
systems in the nation conform to the appropriate standards, such that where interconnects could 
be made, or communication infrastructure is shared, the system concerned will integrate with 
other ITS components. 


7.2. ITS Architecture Levels 


The ITS National Architecture is broken down into several levels, from conceptual to the 
equipment that is placed in the ground. Each of these levels is defined below. 


7.2.1. Logical Architecture 


“The Logical Architecture” presents a functional view of the ITS user services. This perspective 
is separated from those for implementations and physical interface requirements. It defines the 
functions or process specifications that are required to perform ITS user services, and the data 
flows that need to be exchanged between these functions (USDOT, 2007).” This essentially 
provides a framework for how different user services may work together and the general 
communication connections involved. 


7.2.2. Physical Architecture 


The physical architecture shows how different subsystems (i.e., travelers, field elements, centers, 
and vehicles) may be interconnected. The physical architecture is not a detailed design, but 
shows the current and potential communication links so that systems can be integrated and share 
communications infrastructure. From a high level, the physical architecture is commonly 
represented by the “sausage diagram” (Figure 7.1). 
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Figure 7.1: Physical Architecture (Source: USDOT 2007). 


Vile User Services 


To avoid deploying technology for technology’s sake, it is important to start with a properly 
defined need. User services define the purpose of a system from the user’s level. ITS 
architectures are broken into user service bundles like traffic management and commercial 
vehicle operations. Within each of those bundles there are specific user services, which define 
how the ITS system will perform for each specific user. User services are broken down even 
further to market packages. Right now the user service for animal detection systems is not 
explicitly considered in the National ITS Architecture. An example of a user service defined in 
the National ITS Architecture standards is the intersection collision avoidance user service: 


3.6.3.3. Service Description 


The intersection collision warning and control service is specifically aimed at 
providing vehicle operators with assistance in avoiding collisions at intersections. 
The situations addressed include those that arise when vehicles improperly violate 
the right-of-way of another vehicle, or when the right-of-way is not clear. The 
service will provide warnings of imminent collisions with crossing traffic, as well 
as warnings of stop control—either a stop sign or a traffic signal—in the 
intersection ahead. 


There are many diverse causal factors involved in intersection collisions. Among 
the most common of these are driver inattention, failure to obey traffic control 
devices (red signal indications and stop signs), attempting to beat the yellow 
phase of traffic signals, proceeding against cross traffic due to faulty perception 
and obstructed view, and driver intoxication. A variety of countermeasures may 
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be devised depending on both crash type (SCP, CLT or other) and intersection 
traffic control type (signalized, stop signs, and uncontrolled). 


3.6.3.4. Operational Concepts 


The function of this service is to track the position and state of vehicles within a 
defined area surrounding an intersection. The systems may involve infrastructure- 
to-vehicle and/or vehicle-to-vehicle communications. For example, if a vehicle is 
waiting to cross a high-speed roadway, the driver of the crossing vehicle could be 
alerted when there is high-speed traffic approaching. In turn, once a vehicle 
begins crossing the intersection, the other vehicles could be warned and/or 
controlled to avoid a possible collision. One important operational approach is the 
Cooperative Intersection Collision Avoidance System, or CICAS. This system 
would include both vehicle-to-vehicle and vehicle-to-infrastructure links, 
incorporating both one-way and two-way communications. These 
communications technologies would provide various coverage zones and ranges. 
Several media can be used for this purpose, including spread spectrum, 
microwave, millimeter wave, and infrared. 


A type of vehicle-based countermeasure for SCP collisions could utilize video 
and digital image processing to recognize traffic signs and signals and advise or 
warn the driver to stop the vehicle before it encroaches into the intersection in an 
unsafe manner. This concept could be integrated with the In-Vehicle Signing 
subservice, which is part of the En Route Driver Information user service (Section 
3.1.2). In addition, several systems developed primarily for other collision 
categories, such as head-on collision warning and control, may also be useful in 
intersection collision situations. For additional information on HDS, please refer 
to Longitudinal Collision Avoidance (see Section 3.6.1)” (USDOT, 2005). 


7.2.4. | Market Packages and Equipment Packages 


Market packages define the main components of an ITS application and how they are 
interconnected for possible implementation. In a sense, they define the physical architecture for a 
user service. Most user services have a market package, but there is not always a straight one-to- 
one relationship between user services and market packages. Again an example is provided from 
the National ITS Architecture—the intersection safety warning market package. 


“This market package will determine the probability of a collision in an equipped 
intersection (either highway-highway or highway-rail) and provide timely 
warnings to drivers in response to hazardous conditions. Monitors in the roadway 
infrastructure assess vehicle locations and speeds near an intersection. Using this 
information, a warning is determined and communicated to the approaching 
vehicle using a short range communications system. Information can be provided 
to the driver through the market package ATIS9--In-Vehicle Signing” (USDOT 
2002). 


Figure 7.2 shows the schematic for the example market package. “Roadway” and “vehicle” are 
subsystems (note that both of these are in the overarching physical architecture depicted in 
Figure 7.1). The white text boxes within the subsystems describe the “equipment packages” or 
ITS elements needed within these subsystems for this market package to operate. “Basic 
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vehicle,” “potential obstacles,” and “driver” are external elements that participate in the market 
package. The arrows show the information flows. 


intersection status 


Figure 7.2: Market Package Example (Source: USDOT, 2002). 


7.3. Purpose 


The purpose of this chapter is to define how the systems used in the test bed for animal detection 
systems at the TRANSCEND facility near Lewistown relate to the National ITS Architecture. A 
regional ITS architecture was previously developed and summarized in two reports, which will 
be relied upon heavily in this report. The first report is the Greater Yellowstone ITS (GRYITS) 
report (Ice & Associates, 1999) which provided an ITS architecture for the region in and around 
Yellowstone National Park. The second is Montana Regional Architecture (Strong & Eidswick, 
2005). 


The intent of this chapter is not to recreate a regional architecture, but summarize and update the 
architecture elements that relate to animal detection systems. The following summarizes the 
purpose of the regional architecture. 


“A regional architecture for the Greater Yellowstone Rural ITS Priority Corridor 
can guide ITS deployments in the region in a manner that is compatible with 
national efforts including the National ITS Architecture and ITS standards. 
Benefits of a regional architecture that demonstrates conformance with the 
National ITS Architecture include: 


e Section 5206(e) of the Transportation Equity Act for the 21° Century (TEA- 
21) requires that ITS projects using funds from the Highway Trust Fund 
conform to the National ITS Architecture and standards. The GYRITS 
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regional architecture presented in this report meets or exceeds the interim 
policy guidance for architecture conformity from US DOT. It is anticipated 
that this report will also support the final conformity policy that will be 
published later in 1999. 


e A regional architecture facilitates regional integration. It helps agencies and 
other stakeholders to identify and plan for the many integration and 
information sharing opportunities which ITS offers. 


e A regional architecture that conforms with the National ITS Architecture and 
identifies ITS standards enables other ITS systems that will be developed for 
use throughout the U.S. to operate in the Yellowstone region. The regional 
architecture and project implementation guidance provided in this report 
addresses this national interoperability objective. 


e Transportation improvements in the region will be made one project at a time. 
A regional architecture provides guidance for how these projects should fit 
together, improving interoperability between the projects, making efficient use 
of scarce resources, and facilitating future ITS expansion in the region.” (Ice, 
1999). 


7.4. Incorporating RADS into the National ITS Architecture 


This section provides detail on how animal detection systems fit into the regional architecture 
including the physical architecture, the user service, the market package, and one equipment 
package. Since the user service is not likely to be combined with other user services within the 
ITS architecture, the logical architecture is not discussed. 


7.4.1. Physical Architecture 


Figure 7.3 shows the physical architecture for the region as defined in the GYRITS architecture. 
Notice “animal vehicle warning system” is depicted in the roadside subsystem. The physical 
architecture provides flexibility of communication infrastructure (wireless or wireline) 
depending on the specific implementation. 
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Figure 7.3: Sausage Diagram for GYRITS Regional Architecture (Source: Ice, 1999). 


Ta. User Service 


Animal detection systems provide drivers with a warning when an animal is on or near the 
roadway. A warning that large animals are on or near the road may result in fewer and less 
severe collisions. The need for animal detection systems, or for a reduction in collisions with 
large mammals, is based on the following facts: 


e There are an estimated 1—2 million collisions with large mammals (deer and larger) in the 
United States per year, and their number is increasing (Huijser et al., 2007). 


e Although “only” 4.6 percent of all reported animal—vehicle collisions result in human 
injury, and “only” 0.4 percent result in human fatality (Huijser et al., 2007), collisions 
with large mammals amount to an estimated 29,000 human injuries and 211 human 
fatalities in the United States per year (Conover et al., 1995). 


e Collisions with large mammals are costly. The costs associated with the average deer-, 
elk—, and moose—vehicle collision is estimated at US$6,617, US$17,483, and US$30,760, 
respectively (Huijser et al., submitted). These cost estimates include the following 
components: vehicle repair, human injuries, human fatalities, towing, accident attendance 
and investigation, hunting value of the animal, and carcass removal and disposal. 
Assuming that the vast majority of collisions with large mammals relate to deer (Huijser 
et al., submitted), the total costs associated with large mammal—vehicle collisions is 
estimated at US$6,617,000,000-US$13,234,000,000 per year. 
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7.4.3. Market Package 


The market package for animal detection systems was previously developed for the Montana 
Regional ITS Architecture. 


“Encroachment of animals on the roadway is a significant problem in rural areas in the United 
States. The Animal—vehicle Detection Market Package combines sensors that detect animals with 
a dynamic warning system that warns drivers of the animal’s presence on or near the roadway. 
While early implementations are likely to operate autonomously, future implementations may 
allow remote status reporting and calibration of the system to facilitate fault detection and 
maintenance of these potentially remote systems” (Strong & Eidswick, 2005). 


Figure 7.4 shows the communication link between the roadside and a center, consistent with the 
national architecture. The system could be monitored by a traffic management center. The basic 
system is envisioned to warn the driver through roadside signing. If in-vehicle warning systems 
become more prevalent, the system could communicate with the vehicle to provide in-vehicle 
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Figure 7.4: Animal detection system Market Package. 


Figure 7.5 shows how each piece of the animal detection system equipment package, inside the 
dashed line, connects to each other and how the entire equipment package connects with the 
other elements of the market package. Within the equipment packages there is a wide variety of 
equipment to use, and different ways to get the information to the driver on the road. For 
example the detection system could be a motion sensor with infrared to detect the animals, or it 
could use a radar detection system. The detection system will provide data to the master 
controller. The data can be in terms of actual presence detection, or a raw signal that is processed 
by the master controller. The master controller must activate the warning sign when a presence is 
detected. A data storage unit is optional, but useful for evaluating and improving the system. The 
data storage unit will record the time and location of detections for future download and analysis. 
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Figure 7.5: Potential equipment package. 


7.5. Standards 


Ideally, a single vendor will provide the entire system, in which case standards and requirements 
related to National ITS Architecture only pertain to the external communication links (the arrows 
in Figure 7.5 that cross the dotted line of the equipment package boundary). However, 
commonly these systems are delivered by several vendors, and a systems integrator must make 
all the pieces work together. The standards discussed in regard to the external communication 
links (master controller to management center) are highly important. Additional standards would 
allow interchangeability (use of several vendor products) within a single system. Depending on 
the exact nature of the detection system, standards between devices within the animal detection 
system equipment package may not be viable. 


7.6. Master Controller to Management Center 


The reality is that most locations where animal detection systems would be beneficial are in rural 
locations with limited power and communication. The exact system used to communicate 
between the Traffic Management Center and the master controller (e.g., satellite, cell phone, 
fiber, twisted pair) should be determined based on cost effectiveness for the specific location. 


It is strongly encouraged that as a general specification, the animal detection system equipment 
package follow the National Transportation Communications for ITS Protocol (NTCIP). Of 
primary importance is the communication link between the master controller and the Traffic 
Management Center. This should follow the NTCIP communication protocols for “center-to- 
field” communication. 


Because the bandwidth is not anticipated to be significant, Simple Network Management 
Protocol (SNMP) is recommended as the NTCIP communication standard. This standard allows 
more flexible implementation, but is not recommended for high-bandwidth applications. 


For more information on NTCIP refer to www.ntcip.org. 
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8. RECOMMENDED PERFORMANCE REQUIREMENTS 


Authors: Marcel P. Huijser & Tiffany D. Holland, Western Transportation Institute, College of 
Engineering, Montana State University 


8.1. Introduction 


Currently there are no generally agreed upon performance requirements for the reliability and 
effectiveness of animal detection systems. This chapter investigates the expectations of different 
stakeholders with regard to system reliability and effectiveness. 


8.2. Methods 
8.2.1. Stakeholders 


The researchers surveyed employees of transportation agencies, natural resource management 
agencies, and the public. A minimum of one employee of each state or provincial transportation 
agency and each state or provincial natural resource management agency in the United States and 
Canada was asked to fill out a survey. The survey was also sent to employees of federal or 
natural transportation and natural resource management agencies in the United States and 
Canada. This survey was also sent to vendors and manufacturers of animal detection systems, 
non-governmental organizations and researchers involved with animal detection systems, but the 
number of respondents of these groups was too low (eight or less per group) to allow for 
inclusion in the analyses. In addition, the researchers surveyed the traveling public along a road 
section equipped with an animal detection system (US Hwy 191 between Big Sky and West 
Yellowstone, Montana) (see also Huijser et al., 2009). The surveys were anonymous, though 
respondents from agencies were asked to indicate what country their organization is based in. 


8.2.2. Survey 


The questions presented to transportation agencies and natural resource management agencies 
were formulated slightly differently from the questions presented to the public: 


e Question 1: 


For agencies: Animal detection systems are designed to warn drivers when animals are on or 
near the roadway. For your organization to implement such a system, approve of its 
installation or use, or to sell such a system, what percentage of large animals (deer and 
larger) that approach the road do you think should be detected by an animal detection 
system? 

For public: Animal detection systems are designed to warn you when animals are on or near 
the roadway. For you to be confident in such a system, what percentage of large animals 


(deer and larger) that approach the road do you think should be detected by an animal 
detection system? 


Potential answers: 60% or less, 61-70%, 71-80%, 81-85%, 86-90%, 91-95%, 96-99%, 
100% (all large animals that approach the road are detected), I do not know, I do not wish to 
answer this question. 
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e Question 2: 


For agencies: Depending on the technology used, certain weather conditions, low-flying 
birds, falling leaves or high vegetation can result in a “detection” and the activation of the 
warning signs. For your organization to implement an animal detection system, approve of its 
installation or use, or to sell such a system, what percentage of the total number of detections 
would you allow to be “false” (that is, the warning lights are on, but there is not really a large 
animal present)? 


For public: Certain weather conditions, low-flying birds, falling leaves or high vegetation can 
result in a “detection” and the activation of the warning signs. What percentage of the total 
number of detections would you allow to be “false” (that is, the warning lights are on, but 
there is not really a large animal present)? 


Potential answers: 41% or more, 31-40%, 21-30%, 11-20%, 6-10%, 1-5%, 0% (the 
warning signs are only activated when a large animal is really there), I do not know, I do not 
wish to answer this question. 


e Question 3: 


For agencies: For your organization to implement an animal detection system, approve of its 
installation or use, or to sell such a system, what percentage reduction in collisions with large 
wildlife (deer and larger) would you want to see or expect to see as a result of the presence of 
an animal detection system? 


For public: What percentage reduction in collisions with large wildlife (deer and larger) 
would you want to see or expect to see as a result of the presence of an animal detection 
system? 


Potential answers: 60% or less, 61-70%, 71-80%, 81-85%, 86-90%, 91-95%, 96-99%, 
100% (all wildlife—vehicle collisions are prevented), I do not know, I do not wish to answer 
this question. 


The survey for agencies was web-based and was conducted between February 20, 2008 and June 
9, 2008. Employees of the agencies were sent an e-mail with an introductory letter and a link to 
the web site with the survey. The survey for the public was also web-based. The public was 
made aware of the survey in three different ways: 


e Flyers with the web site address were distributed at gas stations and other locations in 
West Yellowstone and Big Sky (5 locations in each town). The flyers contained a brief 
background of the project and a link to a web site for the survey. 


e Direct surveys were conducted at a gas station in Big Sky (Figure 8.1). If travelers 
wanted to participate in the survey but did not have time to complete the survey at that 
time, they were provided with the option to fill out the survey on the web site or fill out a 
hard copy of the survey and return it by mail. 


e The web site for the survey was advertised in local and regional media. 


The survey for the public was conducted between August 24, 2007 and August 3, 2008. 
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——————— 
Did you see the animal 
detection system?. 


Figure 8.1: Booth set up for taking surveys at a gas station in Big Sky, Montana (Photo: Angela 
Kociolek, WTI/MSU). 


8.2.3. Data analyses 


Responses from the three stakeholder groups were investigated to recommend minimum 
performance requirements for the reliability and effectiveness of animal detection systems. The 
researchers calculated cumulative percentages for the number of respondents from each 
stakeholder group for each potential answer to a question, starting from the category with the 
lowest requirement. For example, for question 1, the percentage of respondents that selected 60% 
or less was calculated. Then the percentage of respondents who selected 60% or less or 61-70% 
was calculated, and so on, until the last category (100%), which brought the cumulative 
percentage to 100% by definition. Respondents who selected “I do not know,” “I do not wish to 
answer this question,” or that chose to not answer a question at all were excluded from the 
analyses. The calculations were carried out for each of the three questions and for each of the 
three stakeholder groups, and the results were plotted in graphs. The researchers recommend that 
the minimum performance requirements for the reliability and effectiveness of animal detection 
systems should satisfy the majority (50% or more) of each of the three stakeholder groups. 


8.3. Results 


The number of respondents was similar for transportation agencies and natural resource 
management agencies, while the number of respondents from the public was much greater (Table 
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8.1). Most of the respondents from agencies were from the United States; relatively few were 
from Canada. The public survey did not ask for country information, so that breakdown is not 


known. 


Table 8.1: The number of respondents by group and location. 


Stakeholder group 


Transportation agencies 
Natural resource management agencies 


Public 


United 
States 


37 


35 


Unknown 


Canada 


Unknown 


Total 


43 


37 


160 


The majority (=50%) of all three stakeholder groups would be satisfied with animal detection 
systems detecting 91-95% or more of all large animals that approach the road (Figure 8.2). 
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Figure 8.2: The percentage of large animals approaching the road that survey respondents said 
should be detected by animal detection systems. The number of respondents included in the 
analysis was 31 for transportation agencies, 31 for natural resource management agencies, and 

136 for the public. 
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The majority (50%) of all three stakeholder groups would be satisfied with animal detection 
systems for which 6—10% or less of all detections be are false (Figure 8.3). 
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Figure 8.3: The percentage of detections by animal detection systems respondents would allow to 
be false. The number of respondents included in the analysis was 32 for transportation agencies, 
31 for natural resource management agencies, and 132 for the public. 


The majority (=50%) of all three stakeholder groups would be satisfied with animal detection 
systems that provide a reduction of 71-80% or more in wildlife—vehicle collisions (Figure 8.4). 
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Figure 8.4: The percentage reduction in wildlife—vehicle collisions that is considered desirable. 
The number of respondents included in the analysis was 31 for transportation agencies, 32 for 
natural resource management agencies, and 128 for the public. 
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8.4. Discussion and Conclusion 


The stakeholders had considerable agreement in their responses with regard to the reliability and 
effectiveness of animal detection systems. Based on the survey results, researchers recommend 
the following performance requirements for the reliability and effectiveness of animal detection 
systems: 


e Animal detection systems should detect 91-95% or more of all large animals that 
approach the road. 


e Animal detection systems that had a false detection rate (false positives) of 6—10% or less 
would be acceptable. 


e Use of animal detection systems should result in a reduction of 71-80% or more in 
wildlife—vehicle collisions. 


The recommended performance requirements for the reliability of animal detection systems were 
compared to the results of the reliability tests (chapter 4) (Table 8.2). Five of the nine systems 
meet the recommended performance requirements for reliability. Note that the reliability 
performance for the Camrix system was split into before and after system modification as system 
modification had a significant effect on its reliability performance (see Table 5.4, Chapter 5). 


While the researchers recommend performance requirements that would satisfy the majority 
(250%) of all three stakeholder groups, stakeholders may decide to adopt higher (e.g., 80% of all 
respondents agree) or lower (e.g., 40% of stakeholders agree) norms for these performance 
requirements. This can cause additional systems to meet or no longer meet such performance 
requirements for reliability. If and once the stakeholder groups agree on performance 
requirements for the reliability and effectiveness of animal detection systems, agencies or other 
organizations can clearly communicate internally and externally, including to the public and 
other stakeholders, what animal detection systems that may be installed can and cannot be 
expected to do. Furthermore, performance requirements for the reliability of animal detection 
systems provide important guidance for vendors and manufacturers of animal detection systems. 
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Table 8.2: The reliability of each system in relation to the recommended minimum norms. The 
percentage of intrusions detected is similar, though not exactly the same as the inverse of the 
percentage of false negatives (see chapter 4) (‘alternative calculation: 81.2%; “alternative 
calculation: 81.8%; alternative calculation: 75.5%). 


System Manufacturer and ID number False False Intrusions Meets 
number system name positives negatives detected | recommended 
: (%) (all types (%) norms 
(Higure'3-2) combined) (%) (yes/no) 
1 Xtralis 7 0.00 10.29 91.75 Yes 
(ADPRO) 
2 Xtralis 5-6 0.00 20.88 85.43 No 
(ADPRO) 
3 STS 1 0.00 30.91 72.47 No 
(RADS I) 
4 STS 2 0.00 15.94 88.35 No 
(RADS II) 
5 Calonder Energy | 1 0.60 0.48 99.54 Yes 
(CAL 92, LS-WS- 
WE 45) 
6 Calonder Energy | 2 0.00 1.16 98.85 Yes 
(CAL 92, IR-204- 
319/M3) 
7 Camrix Overall 0.07 30.21 89.41"! No 
CEENT) Before mod. 0.07 30.41 89.337 No 
After mod. 0.00 27.00 90.20° No 
8 Xtralis 1-2 0.97 6.53 95.19 Yes 
(ADPRO) 
9 Goodson 0.82 0.00 100.00 Yes 
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9. SITE REVIEWS 


Author: Marcel P. Huijser, Western Transportation Institute, College of Engineering, Montana 
State University 


9.1. Sites 


Seven different road sections in western Montana, USA, were reviewed for the potential 
installation of an animal detection system (Table 9.1). These road sections were suggested by Pat 
Basting and Deb Wambach of the Montana Department of Transportation. It is not the purpose 
of this chapter to select the "best" road section for the potential installation of an animal 
detection system. The purpose is to provide site descriptions so that the Montana Department of 
Transportation has information to make an educated decision for each site individually. 


Table 9.1: Site locations. 


Site Road name Mi reference Length road | Location description 
posts section (km (mi)) 


1 1-90 14.4-16.0 2.6 km (1.6 mi) | DeBorgia West 

2 1-90 19.5-20.7 1.9 km (1.2 mi) | DeBorgia East 

3 1-90 82.5-83.6 1.8 km (1.1 mi) | Ninemile 

4 Hwy 206 1.7-4.0 3.7 km (2.3 mi) | Kalispell 

5 MT Hwy 200 31.1-34.2 5.0 km (3.1 mi) | Clearwater Jct / Blackfoot 


Clearwater Game Range 


6 MT Hwy 83 0.5-4.0 5.6 km (3.5 mi) | Clearwater Jct / Blackfoot 
Clearwater Game Range 


7 I-15 162.0-165.0 4.8 km (3.0 mi) | Boulder 


The road sections are shown in figures 9.1 through 9.7. The images show the location of the mile 
reference posts (ending in ".0") and the approximate tenth of a mile points in between with 
satellite imagery background. Note that these locations are the exact locations where images 
were taken of the road and right-of-way, and sometimes of the surrounding landscape as well 
(See Appendix B). Also note that the tenth of a mile locations are not always a tenth of a mile 
apart as the distance between mile reference posts is not always exactly one mile. 
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Figure 9.1: I-90 DeBorgia West. EB = East Bound; WB = West Bound. 
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Figure 9.2: I-90 DeBorgia East. EB = East Bound; WB = West Bound. 
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Figure 9.3: I-90 Ninemile. EB = East Bound; WB = West Bound. 
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Figure 9.4: Hwy 206 Kalispell. 
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Figure 9.5: MT Hwy 200 Clearwater Jct / Blackfoot Clearwater Game Range. 
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Figure 9.6: MT Hwy 83 Clearwater Jct / Blackfoot Clearwater Game Range. 
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Figure 9.7: I-15 Boulder. NB = North Bound; SB = South Bound. 


9.2. Review Parameters 


Each road section was reviewed with regard to the following parameters, as long as the data were 
available to the researchers. These parameters are partially based on Hujjser et al. (2006a): 


e Animal-vehicle collisions. The site should have a history of a relatively high number of 
animal-vehicle collisions with large animals, especially ungulates (e.g., deer, elk or 
moose). This is for two reasons: 1. the costs associated with the purchase, installation, 
and operation and maintenance of an animal detection system may be compensated by 
the savings associated with reduced animal vehicle collisions, and 2. if an animal 
detection system is evaluated for its effectiveness in reducing animal-vehicle collisions, 
historic data on animal-vehicle collisions should preferably be available (comparison in 
time). In addition, historic animal-vehicle collisions from control sites are helpful 
(comparison in space). 
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e Animal movements. The site should preferably be located in an area where many large 
animals (e.g., deer, elk or moose) are known to cross the road (daily movements or 
seasonal migration). Note: not all animal movements across a road result in animal- 
vehicle collisions. This may protect travelers against potential future animal-vehicle 
collisions and, in addition, the animals that cross the road should, at least theoretically, be 
better protected against potential future collisions with vehicles. 


e Traffic volume and trucks. As traffic volume increases it becomes less and less desirable 
to have large animals cross at grade. In addition, above a certain traffic volume, the 
barrier effect of the road may be close to absolute with few animals that even try to still 
cross the road. In that type of situation, the problem of collisions has been large replaced 
by that of a barrier to animals. Furthermore, large vehicles such as freight trucks may be 
less likely to respond to the activated warning signs because the expense and perhaps also 
risk involved in braking when carrying a heavy load and because large ungulates may not 
cause substantial damage to the vehicle to begin with. 


e Terrain. The terrain must allow for the installation of an animal detection system. For 
example, an abundance of ridges, gullies and rocky outcrops may make a location less 
suitable for an animal detection system, especially break-the-beam systems. Difficult 
terrain may also require more sensors and other equipment than relatively flat areas 
would require. 


e Curves and access roads. The number of curves and access roads should be kept to a 
minimum to minimize the number of sensors and to avoid gaps (blind spots) or excessive 
false positives caused by traffic turning on or off the road, depending on what sets off the 
sensors. 


e Vegetation. The vegetation should allow for the installation of an animal detection 
system. For example, bushes and trees that grow up to the edge of the pavement increase 
the chance of triggering the system, i.e., they would cause excessive false positives for 
most area cover, or break-the-beam systems. 


e Length road section. If an animal detection system is deployed as a stand alone mitigation 
measure, the road section must be at least 805-1609 m (0.5—1.0 mi) long to be able to 
accommodate for potential spatial errors in the location of historic road kill data that was 
used to select the site. If an animal detection system is installed in a gap in a wildlife 
fence, the gap width can be variable, but a gap is typically between 30 and 200 m wide, 
depending on the range of the sensors and the local conditions. 


e Changes in road or landscape. The road and surrounding landscape should not be 
scheduled to undergo major changes within the life span of the mitigation measure; for 
animal detection systems about 10 years. However, should changes in the landscape 
occur and change where animals cross the road and where animals are hit by vehicles, 
then one may consider relocating an animal detection system. Nonetheless, there are 
relocation costs involved for such an effort. In addition, major changes, other than the 
installation of the animal detection system, would confound the results of a potential 
study into the effectiveness of the animal detection system in reducing animal-vehicle 
collisions. 
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e Project partners. All the organizations and individuals that have jurisdiction or that are 
stakeholders in activities at the study site should support the project. This includes 
support for installation, operation and maintenance. 


e Travel costs. The site should preferably be close to where operation and maintenance 
personnel have their offices. This reduces costs for travel and stay. 


e Power. The site should allow for either solar power or a connection to 110 V power 
source. 


e Pull-out. The site should preferably have a safe pull-out location for vendors and 
maintenance and research personnel. 


e Controlled access. The site should preferably have a low risk of theft and vandalism, e.g., 
a controlled access road. 


9.3. Site Conditions 


The site conditions are described in detail in the sections below. Table 9.2 provides a summary 
of the site conditions. 


9.3.1. Animal-vehicle collisions 


Wildlife carcass removal data were obtained from the Montana Department of Transportation to 
estimate the costs associated with wildlife-vehicle collisions on the road sections concerned. 
These data do not include all wildlife-vehicle collisions, as some animals are never found or not 
removed. Thus the data provide a minimum estimate on the number of wildlife-vehicle 
collisions. Only large ungulates (deer, elk, and moose) were included in our analyses, but other 
large mammals (mountain lions, wolves, and black bear were estimated to result in damage 
similar to deer). The costs for the average collision with a deer, an elk, and a moose has been 
estimated at $6,617, $17,483, and $30,760 respectively (Hutjser et al., Submitted). These cost 
estimates were combined with the wildlife carcass removal data to calculate the costs associated 
with deer-, elk-, and moose-vehicle collisions combined per kilometer per year. These costs were 
presented in graphs for the road sections concerned. The graphs also showed the thresholds the 
costs must meet in order to have different mitigation measures generate benefits in excess of 
costs. The thresholds shown are for the following combinations of mitigation measures (see 
Huijser et al., Submitted for details): 


e Animal detection system 
e Fence, gap (once every 2 km), animal detection system in gap, jump-outs 


e Fence, under- and overpass (underpass once every 2 km, overpass once every 24 km), 
jump-outs 


e Fence, under pass (once every 2 km), jump-outs 


At locations where the costs reach or exceed the threshold values, the mitigation measure 
concerned would generate benefits in excess of costs. Note that the road sections were expanded 
on either side by a few miles to accommodate for edge effects (zero values) for the average costs 
per kilometer. 
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I-90 DeBorgia West and I-90 DeBorgia East combined 
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Figure 9.8: I-90 DeBorgia East and West. The costs (in 2007 US$) associated with wildlife- 
vehicle collisions (white-tailed deer, mule deer, elk, moose, and 7 black bears and 1 mountain 
lion (black bears and mountain lions were estimated to have equal cost as deer) along the 4-lane 
I-90 (mi reference posts 13.0-22.0) per year (average 1998-2008), and the threshold values (at 
3% discount rate) that need to be met in order to have the benefits of individual mitigation 
measures exceed the costs over a 75 year long time period. Note that the costs at each 0.1 mi 
concerned and five adjacent 0.1 mi units were summed (0.6 mi = | km) to estimate the costs per 
kilometer. 
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Figure 9.9: I-90 Ninemile. The costs (in 2007 US$) associated with wildlife-vehicle collisions 
(white-tailed deer, mule deer, and 2 black bears and 1 wolf (black bears and wolves were 
estimated to have equal cost as deer) along the 4-lane I-90 (mi reference posts 80.0-85.0) per 
year (average 1998-2008), and the threshold values (at 3% discount rate) that need to be met in 
order to have the benefits of individual mitigation measures exceed the costs over a 75 year long 
time period. Note that the costs at each 0.1 mi concerned and five adjacent 0.1 mi units were 
summed (0.6 mi = | km) to estimate the costs per kilometer. 
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Figure 9.10: Hwy 206 Kalispell. The costs (in 2007 US$) associated with wildlife-vehicle 
collisions (white-tailed deer, mule deer, and 3 mountain lions (mountain lions were estimated to 
have equal cost as deer) along the 2-lane Hwy 206 (mi reference posts 1.0-6.0) per year (average 
2005-2006), and the threshold values (at 3% discount rate) that need to be met in order to have 
the benefits of individual mitigation measures exceed the costs over a 75 year long time period. 
Note that the costs at each 0.1 mi concerned and five adjacent 0.1 mi units were summed (0.6 mi 
= | km) to estimate the costs per kilometer. 
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MT Hwy 200 Clearwater Jct 
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Figure 9.11: MT Hwy 200 Clearwater Jct. The costs (in 2007 US$) associated with wildlife- 
vehicle collisions (white-tailed deer, mule deer, elk, and 2 black bears and 1 mountain lion (black 
bears and mountain lion were estimated to have equal cost as deer) along the 2-lane MT Hwy 
200 (mi reference posts 30.0-36.0) per year (average 1998-2008), and the threshold values (at 
3% discount rate) that need to be met in order to have the benefits of individual mitigation 
measures exceed the costs over a 75 year long time period. Note that the costs at each 0.1 mi 
concerned and five adjacent 0.1 mi units were summed (0.6 mi = | km) to estimate the costs per 
kilometer. 


Western Transportation Institute Page 88 


Reliability of animal detection systems Site reviews 


MT Hwy 83 Clearwater Jct 


$40,000 


A A 
wo oo 
id Ady 
ro) °o 
S S 
3S 3S 


$25,000 


“ 
N 
- 
ro) 
ro) 
ro) 


Wildlife-vehicle collision costs (US$/kmlyr) 
Hf 


0.0 1.0 2.0 3.0 4.0 5.0 6.0 
Mile reference post 


teers Threshold animal detection system 


Threshold fence, gap, animal detection system, jump-outs 


- - - - Threshold fence, under- and overpass, jump-outs 


Threshold fence, under pass, jump-outs 


Figure 9.12: MT Hwy 83 Clearwater Jct. The costs (in 2007 US$) associated with wildlife- 
vehicle collisions (white-tailed deer, mule deer, elk, 1 mountain lion (mountain lion were 
estimated to have equal cost as deer) along the 2-lane MT Hwy 83 (mi reference posts 0.0-6.0) 
per year (average 1998-2008), and the threshold values (at 3% discount rate) that need to be met 
in order to have the benefits of individual mitigation measures exceed the costs over a 75 year 
long time period. Note that the costs at each 0.1 mi concerned and five adjacent 0.1 mi units 
were summed (0.6 mi = 1 km) to estimate the costs per kilometer. 
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Figure 9.13: I-15 Boulder. The costs (in 2007 US$) associated with wildlife-vehicle collisions 
(deer and elk) along the 4-lane I-15 (mi reference posts 160.0-167.0) per year (average 1998— 
2007), and the threshold values (at 3% discount rate) that need to be met in order to have the 
benefits of individual mitigation measures exceed the costs over a 75 year long time period. Note 
that the costs at each 0.1 mi concerned and five adjacent 0.1 mi units were summed (0.6 mi = 1 
km) to estimate the costs per kilometer. 
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9.3.2. Animal movements 


No information is available to the authors of this report. Interviews with people who have local 
experience and knowledge with regard to animal movements is recommended. 


9.3.3. Traffic Volume 
I-90 DeBorgia West 


This is a four lane (two lanes in each direction) interstate. Average Annual Daily Traffic 
(AADT) in 2007: 6,720 at reference post 47.8 (MDT, 2008). Percentage of large trucks in 2007: 
25.5% (MDT, 2008). Average Annual Daily Traffic (AADT) in 2007: 7,570 at reference post 
15.5 (MDT, 2009). 


I-90 DeBorgia East 


This is a four lane (two lanes in each direction) interstate. Average Annual Daily Traffic 
(AADT) in 2007: 6,720 at reference post 47.8 (MDT, 2008). Percentage of large trucks in 2007: 
25.5% (MDT, 2008). Average Annual Daily Traffic (AADT) in 2007: 7,570 at reference post 
15.5 (MDT, 2008). 


I-90 Ninemile 


This is a four lane (two lanes in each direction) interstate. Average Annual Daily Traffic 
(AADT) in 2007: 6,720 at reference post 47.8 (MDT, 2008). Percentage of large trucks in 2007: 
25.5% (MDT, 2008). Average Annual Daily Traffic (AADT) in 2007: 7,370 at reference post 
76.8 (MDT, 2009). 


Hwy 206 Kalispell 


This is a two lane (one lane in each direction) highway. Average Annual Daily Traffic (AADT) 
in 2007: 4,380 at reference post 0.5 (MDT, 2009). 


MT Hwy 200 Clearwater Jct 


This is a two lane (one lane in each direction) highway. Average Annual Daily Traffic (AADT) 
in 2007: 2,854 at reference post 30.8 (MDT, 2008). Percentage of large trucks in 2007: 6.1% 
(MDT, 2008). 


MT Hwy 83 Clearwater Jct 


This is a two lane (one lane in each direction) highway. Average Annual Daily Traffic (AADT) 
in 2007: 2,235 at reference post 0.8 (MDT, 2008). Percentage of large trucks in 2007: 5.1% 
(MDT, 2008). 
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I-15 Boulder 


This is a four lane (two lanes in each direction) interstate. Average Annual Daily Traffic 
(AADT) in 2007: 3,120 at reference post 164.5 (MDT, 2009). 


9.3.4. Terrain 
I-90 DeBorgia West 


There is a large culvert at about reference point 15.7, which coincides with a depression in the 
terrain. However, depending on the species, animals may use the culvert as opposed to crossing 
at grade. There is guard rail associated with the underpass for both travel directions. For the 
eastbound lanes there is guard rail present between reference points 14.4-14.5, 14.9-15.0, and 
15.6-15.8 because of steep slopes and the nearby river. Concrete barriers are present in the 
median between reference point 14.4-14.5. 


I-90 DeBorgia East 


The section is mostly flat. There is one short section of guardrail along the eastbound lanes 
(reference points 20.6-20.7) that is associated with a slope. 


I-90 Ninemile 


This section climbs up towards the west, but the right-of-way does not have major rises or 
depressions. A guard rail and, mostly concrete barriers, are present in the median for most of the 
road section (reference posts 8 2.5-83.4). The westbound lanes have guardrail and a steep slope 
at 83.6. The eastbound lanes have guardrail and slopes between reference points 82.7 and 83.2, 
and at 83.6. These are all associated with steep slopes. 


Hwy 206 Kalispell 


The road section between reference points 1.7 and 2.4 was recently re-contoured. However, the 
road section between reference points 2.4 and 4.0 is relatively uneven because of access roads or 
farm land entries. There are two substantial hills in this road section (around reference points 1.8 
and 2.8). 


MT Hwy 200 Clearwater Jct 


This road section is mostly flat. Exceptions are the bridge across the Clearwater river (reference 
point 31.3), and a rise in a curve towards the east end (around reference point 33.5). 
Furthermore, there is a short section of guardrail on both sides of the road at the bridge across the 
Clearwater river (reference point 31.3), and there is a guardrail at the east end (reference point 
14.2) because of a slope. 
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MT Hwy 83 Clearwater Jct 


This road section is mostly flat. Exceptions are slopes adjacent to the southbound lane at 
reference point 2.3 and 4.0. 


I-15 Boulder 


This road section is mostly flat. Exceptions are a hill at reference points 163.6-163.7, and two 
bridges for the Boulder river for the northbound lanes at reference points 162.0 and 163.0 and at 
a bridge at reference point 164.1 (both travel directions). For the northbound lanes guard rails 
and associated slopes are present at reference points 162.0, 162.0-162.2 (median), 162.4-162.5 
(median), 162.9-163.1 (median), 163.0-163.1, 163.2-163.5, 164.1 (both sides). For the 
southbound lanes guard rails and associated slopes are present at reference points 164.2 (both 
sides), 163.3-163.5, and 162.0-162.4 (median). 


9.3.5. Curves and Access Roads 


I-90 DeBorgia West 


The westbound lane has two curves whereas the eastbound lane is straighter and has only one 
curve. Furthermore, this road section is characterized by a wide median with on and off ramps 
(four in total) for a truck weigh station in the median. There are no on or off ramps on the 
"outside" of the road section, but there is an access road to a gravel pit that connects to the 
westbound lane at about reference point 14.95. There are two turn around points for cars in the 
median (restricted access) at about reference point 14.6 and 15.8. 


I-90 DeBorgia East 


Both the westbound and eastbound lanes are straight. Furthermore, this road section is 
characterized by a relatively wide median, mostly with trees and shrubs. There are no on or off 
ramps. There is turn around point for cars in the median (restricted access) at about reference 
point 19.6. 


I-90 Ninemile 


Both the westbound and eastbound lanes are curvy at the beginning and end. Furthermore, this 
road section is characterized by the absence of a median. There is an on and off ramp on the far 
west side (reference points 82.5-82.6). 


Hwy 206 Kalispell 


This road section is straight. However, there are six cross roads or drive ways on the east side, 
and 16 on the west side. 
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MT Hwy 200 Clearwater Jct 


This road section is mostly straight except for a couple of curves close to either end. However, 
there are 15 cross roads or drive ways on the east side, and eight on the west side. 


MT Hwy 83 Clearwater Jct 


This road section is mostly straight, but it has a few slight curves and a more substantial curve at 
reference point 3.5. However, there are nine cross roads, drive ways, or pull-outs on the east side, 
and four on the west side. 


I-15 Boulder 


This is a relatively curvy road section with a wide median, especially between reference points 
162.0-163.0 where the Boulder river is located in the median. There are on and off ramps at 
164.7 and just north of 165.0. There are turn around points for cars in the median (restricted 
access) at about reference point 163.1 and164.3. 


9.3.6. Vegetation 
I-90 DeBorgia West 


The first few meters (yards) from the edge of the pavement are dominated by grasses and herbs. 
Trees and shrubs are far enough from the edge of the pavement to allow for the installation of 
break-the-beam systems. 


I-90 DeBorgia East 


The first few meters (yards) from the edge of the pavement are dominated by grasses and herbs. 
Trees and shrubs are far enough from the edge of the pavement to allow for the installation of 
break-the-beam systems. 


I-90 Ninemile 
The vegetation in the right-of-way is dominated by grasses and herbs. 


The first few meters (yards) from the edge of the pavement are dominated by grasses and herbs. 
Trees and shrubs are far enough from the edge of the pavement to allow for the installation of 
break-the-beam systems. 


Hwy 206 Kalispell 


The first few meters (yards) from the edge of the pavement are dominated by grasses and herbs. 
Trees and shrubs come close to the road in some sections (e.g. around reference points 3.7-3.9). 
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MT Hwy 200 Clearwater Jct 


The vegetation in the right-of-way is dominated by grasses and herbs. Trees and shrubs are 
scarce and only come close to the road at the bridge across the Clearwater (reference point 31.3). 


MT Hwy 83 Clearwater Jct 


The vegetation in the right-of-way is dominated by grasses and herbs. Trees and shrubs are 
scarce, except for the north end. 


I-15 Boulder 


The vegetation in the right-of-way is dominated by grasses and herbs. The northern road section 
is mostly open, but trees and shrubs are closer to the road where the Boulder river runs in the 
median. 


9.3.7. Length Road Section 
See Table 9.1. 


9.3.8. Changes in Road or Landscape 


If implementation of an animal detection system is considered at a road section, other changes in 
the road and landscape should be evaluated, especially if the system is to be evaluated for its 
effectiveness in reducing animal-vehicle collisions. 


9.3.9. Project Partners 


No information is currently available on potential partners for the potential implementation of 
the animal detection systems. 


9.3.10. Travel Costs 


No information is currently available on who would be responsible for operation and 
maintenance of the animal detection systems, should any be installed. 


9.3.11. Power 
I-90 DeBorgia West 


There are power lines at varying distance (a few hundred feet to perhaps a few hundred yards) 
north of the road section. 
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I-90 DeBorgia East 


There are power lines at a few dozen feet on the north side of the westbound lane, just north of 
the frontage road. In addition, there are larger power lines that parallel the eastbound lanes on the 
south side a few hundred feet from the edge of the pavement. 


I-90 Ninemile 


There are power lines that cross the road at reference point 83.3. 


Hwy 206 Kalispell 


There are power lines that parallel the road between reference points 1.7-2.9. There are street 
lights present between reference points 3.6-4.0. 


MT Hwy 200 Clearwater Jct 


Power lines come close north of the road at reference point 32.1, cross the road at reference point 
31.8, and run parallel to the road on the south side until reference point 33.2. 


MT Hwy 83 Clearwater Jct 


Power lines run parallel to the west side of the road. Additionally, power lines cross the road at 
reference point 4.0. 


I-15 Boulder 


Power lines cross the road at reference point 162.0, run parallel to the east side of the road 
between reference points 162.0-163.0, cross the road at 163.0, 164.2-164.3, and run parallel to 
the west side at a distance of a few hundred meters (yards). 


9.3.12. Pull-out 
I-90 DeBorgia West 


There is safe stopping and parking everywhere because of the wide shoulder and clear zone, 
except in road sections that have guard rail. 


I-90 DeBorgia East 


There is safe stopping and parking everywhere because of the wide shoulder and clear zone, 
except in the one short road section that has a guard rail. 
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I-90 Ninemile 


There is safe stopping and parking along most of the westbound lanes because of the wide 
shoulder and clear zone, except in the one short road section that has a guard rail. There is a 
guard rail along most of the eastbound lanes, making it unsuitable for frequent or long stops. 


Hwy 206 Kalispell 


There is safe stopping and parking between reference points 1.7 and 2.4 as this section was 
recently re-contoured. However, it is generally not safe to stop frequently and long between 
reference points 2.4 and 4.0 because of the narrow shoulders and slopes. Furthermore, there are 
two substantial hills in this road section (around reference points 1.8 and 2.8) that limit the sight 
distance to drivers. 


MT Hwy 200 Clearwater Jct 


There is safe stopping and parking along most of the road section (wide shoulders, clear zone, 
short grass-herb vegetation). 


MT Hwy 83 Clearwater Jct 


There is safe stopping and parking along most of the road section (wide shoulders, clear zone, 
short grass-herb vegetation). 


I-15 Boulder 


There is safe stopping and parking everywhere because of the wide shoulder and clear zone, 
except in road sections that have guard rail. 


9.3.13. Controlled Access 


I-90 DeBorgia West 


Drivers are unlikely to stop in this road section, except for truck drivers that must stop at the 
weigh station in the median. Unless a detection system is also installed at the median, people are 
unlikely to access a potential animal detection system. The access road to the gravel pit 
(westbound lanes, mile reference post 14.95) as well as the frontage road that parallels the 
westbound lanes between mile reference posts 15.5 and 16.0, may cause a few users to 
investigate a potential animal detection system though. 


I-90 DeBorgia East 


Drivers are unlikely to stop in this road section. The frontage road that parallels the westbound 
lanes may cause a few users to investigate a potential animal detection system though. 
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I-90 Ninemile 


Drivers are unlikely to stop in this road section, except for the on and off ramps at reference 
point 82.5. 


Hwy 206 Kalispell 


Drivers are unlikely to stop, especially between reference points 2.4 and 4.0 because of the 
narrow shoulders and slopes. However, because of nearby houses, people are likely to investigate 
potential equipment, especially between reference points 1.7-3.0 and 3.6-4.0. 


MT Hwy 200 Clearwater Jct 


Drivers are unlikely to stop, except at the junction with MT Hwy 83 (gas station, rest stop). 
Other road sections where people may investigate the equipment are at driveways and access 
roads, particularly at reference points 33.8-33.9. 


MT Hwy 83 Clearwater Jct 


Drivers are unlikely to stop, except at the junction with MT Hwy 83 (gas station). Other road 
sections where people may investigate the equipment are at driveways, access roads, and pull- 
outs. 


I-15 Boulder 


Drivers are unlikely to stop at this road section. 
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Table 9.2: The suitability of the road sections with regard to the installation of an animal 
detection system. + + + = strongly suitable; —-—— strongly not suitable or a severe concern. 
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10. DISCUSSION AND CONCLUSIONS 


Author: Marcel P. Huijser, Western Transportation Institute, College of Engineering, Montana 
State University 


10.1. Discussion 


The results of the reliability tests showed that different detection technologies detect large 
animals more or less frequently as an animal passes through the detection area or line of 
detection. The percentage of false positives and the average number of false positives per hour 
was relatively low for all systems (<1%; <0.10/hr). The percentage of false negatives (all types 
of false negatives combined) and the average number of false negatives per hour was highly 
variable (0O-31%; 0—1.61/hr)). The percentage of intrusions (i.e., animal movements across the 
detection line) that were detected varied between 73% and 100%. The results suggest that some 
animal detection systems are quite reliable in detecting large mammals with few false positives 
and false negatives, whereas other systems suffer from relatively many false negatives. 


The reliability of animal detection systems is influenced by a range of environmental conditions. 
High winds were associated with an increase in different types of false negatives for most 
passive infrared area-cover systems. High winds were associated with both an increase in false 
positives and a decrease in false positives for different types of systems, suggesting that passive 
infrared area-cover systems become less sensitive with high winds whereas break-the-beam 
systems that rely on a very narrow beam may start generating false positives, presumably 
because the sensors sway slightly in and out of alignment. The latter suggests the importance of a 
stable foundation and pole for break-the-beam systems. Stable foundations and poles may also be 
beneficial to passive infrared area-cover systems, but it is unclear if the increase in false 
negatives for such systems is caused by movement of the sensors, which tend to be higher up on 
a pole than sensors for break-the-beam systems, or by vegetation or pockets of hot and cold air 
that move in the wind. The effects of wind direction are hard to interpret, but it may be that 
winds oriented perpendicular to the systems caused vegetation or pockets of hot and cold air to 
trigger systems more often than winds oriented more parallel to the systems. Higher temperatures 
are generally associated with higher error rates. This could be due to temperature causing 
reduced performance of the equipment. In addition, passive infrared systems may not be able to 
distinguish clearly between pockets of hot air and moving animals. However, higher 
temperatures are concentrated in time (summer) and it is possible that factors other than 
temperature caused more errors in summer. Animal behavior and possible effects on the 
likelihood of correct detections and errors may have also been influenced by temperature. Three 
systems had fewer false negatives during the night compared to during the day. This may be 
related to lower temperatures or higher contrasts in temperatures between the animals and their 
surroundings during the night. However one system had more false negatives during the night 
compared to during the day. Excellent visibility was associated with fewer false positives for a 
break-the-beam system, which suggests that relatively low visibility may block or reduce the 
narrow signal path of optical break-the-beam systems. It is unclear why excellent visibility may 
have caused another system to increase a particular type of false negatives. Precipitation was 
rarely observed during the test periods and its effect on system reliability is unclear. However, 
higher relative humidity was mostly associated with an increase in errors, and to a lesser extent 
with a decrease in errors. Finally, llamas were substantially harder to detect for most systems, 
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especially passive infrared area-cover systems, than horses, probably because of their smaller 
body size. 


Three stakeholder groups—, employees of transportation agencies, employees of natural 
resource management agencies, and the traveling public—were surveyed with regard to their 
expectations on the reliability and effectiveness of animal detection systems. There was 
considerable agreement in the responses of the three stakeholders. The researchers recommend 
the following performance requirements for the reliability and effectiveness of animal detection 
systems: 


e Animal detection systems should detect 91-95% or more of all large animals that 
approach the road. 


e Animal detection systems that had a false detection rate of 6—-10% or less would be 
acceptable. 


e Use of animal detection systems should result in a reduction of 71-80% or more in 
wildlife—vehicle collisions. 


The recommended performance requirements for the reliability of animal detection systems were 
compared to the results of the reliability tests. Five of the nine systems met the recommended 
performance requirements for reliability. However, experiences with installation, operation and 
maintenance show that the robustness of animal detection systems may have to be improved 
before the systems can be deployed on a large scale. 


This report also presented a concept of operation and a review of ITS architecture and 
infrastructure for animal detection systems. Currently, roadside animal detection systems present 
drivers with warnings displayed on road signs. In the future, roadside animal detection systems 
may also transmit warning signals to traffic approaching a location where a large animal has 
been detected on or near the road. This procedure would require a two-way GPS-based 
communication system. With animal detection system deployments becoming more numerous, 
standards for communication and ITS integration will have to be further developed and accepted. 


10.2. Conclusions 


e Different detection technologies detect large animals more or less frequently as an animal 
passes through the detection area or line of detection. This implies that care must be taken 
in evaluating the reliability of different technologies, and in comparing systems or 
minimum performance requirements. 


e The percentage of false positives and the average number of false positives per hour was 
relatively low for all systems (<1%; <0.10/hr). False positives do not appear to be a major 
concern with regard to the reliability of animal detection systems. 


e The percentage of false negatives (all types of false negatives combined) and the average 
number of false negatives per hour was highly variable (0-31%; O—1.61/hr). The 
percentage of intrusions (1.e., animal movements across the detection line) that were 
detected varied between 73% and 100%. The results suggest that false negatives are a 
major concern for some animal detection systems, but not for others. 


e Environmental conditions and the target species influence the reliability performance of 
animal detection systems. Therefore the environmental conditions at a site and te taret 
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species should be carefully evaluated before selecting a suitable system. Besides system 
reliability, system robustness, and the road length that the sensors are able to cover needs 
to be considered. 


e The recommended performance requirements for the reliability of animal detection 
systems were compared to the results of the reliability tests. Five of the nine systems met 
the recommended performance requirements for reliability. However, experiences with 
installation, operation, and maintenance show that the robustness of animal detection 
systems may have to be improved before the systems can be deployed on a large scale. 


e Currently, roadside animal detection systems present drivers with warnings displayed on 
road signs. In the future, roadside animal detection systems may also transmit warning 
signals to traffic approaching a location where a large animal has been detected on or 
near the road. With animal detection system deployments becoming more numerous, 
standards for communication and ITS integration will have to be further developed and 
accepted. 
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12. APPENDIX A: CONTACT DETAILS FOR MANUFACTURERS AND 
LIVESTOCK SUPPLIER 
Systems 1, 2 and 8 
Xtralis 
700 Longwater Drive #100 
Norwell, MA 02061 
Main contact: Steve Stettner 
Phone: 781-740-2223, Fax: 781-740-4433, 
E-mail: sstettner@xtralis.com 


Web site: http://www.xtralis.com 


Systems 3 and 4 

ICx Radar Systems 

(formerly Sensor Technologies and Systems, Inc.) 
“RADS (Roadside Animal Detection System)” 
8900 East Chaparral Road 

Scottsdale, AZ 85250, USA 

Main contact: Walker Butler 

E-mail: walker.butler@icxt.com 

Phone: 480-483-1997, Fax: 480-483-2011 


Web site: http://radarsystems.icxt.com 


Systems 5 and 6 

Calonder Energy 

“CAL 92” 

Willy Berchtold 

1436 Van Asche Drive 

Fayetteville, AR 72704, USA 

Phone: 479-521-0056, Fax: 479-521-9116 


E-mail: info@calonderenergy.com 


Web site: www.calonderenergy.com 
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System 7 

Camrix 

“A.L.E.R.T. (Animal Location and Evasive Response Technology)” 
Main contact: Mike Doyle 

1139 Holly Dr. 

Bozeman, MT 59715, USA 

Phone: 406-209-1928 (cell) 


System 9 

Goodson and Associates, Inc 

10614 Widmer 

Lenexa, KS 66215, USA 

Main contact: Bill Goodson 

Phone: 1-800-544-5415 / 913-345-8555, Fax: 913-345-8555 


E-mail: bgoodson@trailmaster.com 


http://www.trailmaster.com/company.php 


IR cameras 

Fuhrman Diversified, Inc. 

2912 Bayport Blvd. 

Seabrook, TX 77586-1501, USA 

Main contact: Richard Fuhrman 

Phone: 281-474-1388, Fax: 281-474-1390 
E-mail: fdi@flash.net 


Web site: http://www. fieldcam.com/ 


Livestock supplier 

Lethia Olson 

5520 Grinde Rd. 

Lewistown, MT 59457-8044, USA 

Phone: 406-538-5818, Fax: 406-538-2893 
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13.APPENDIX B: PHOTOS FROM THE SITES REVIEWED IN 
CHAPTER 9 


Because: 
1. The number of photos, and thus the file size (1.6 GB) and number of pages involved, and; 
2. The fact that the photos from the site review are only of interest to a few people, 


the photos were submitted to the Montana Department of Transportation (MDT) and the Federal 
Highway Administration (FHWA) on DVD only. 


The DVD with photos from the sites was delivered to the following MDT and FHWA 
employees: 


Name: BALSLEY, Phillard 
Location: TRAFFIC - ELECTRICAL 
City: Helena 

Phone Number: (406) 444-6218 
Email: pbalsley@mt.gov 


Name: BASTING, Pat 

Location: MISSOULA - ENVIRONMENTAL 
City: Missoula 

Phone Number: (406) 523-5872 

Email: pbasting@mt.gov 


Name: SELISKAR, Bob 
Location: FHWA 

City: Helena 

Phone Number: (406) 441-3903 
Email: a0420@mt.gov 


Name: SILLICK, Susan 

Location: HELENA - RESEARCH 
City: Helena 

Phone Number: (406) 444-7693 
Email: ssillick@mt.gov 


Name: WAMBACH, Deborah 

Location: HELENA - ENVIRONMENTAL 
City: Helena 

Phone Number: (406) 444-0461 

Email: dwambach@mt.gov 
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